Linux Print System at Cisco Systems, Inc.

Cisco runs a redundant system of 50 print servers using Linux, Samba and Netatalk. It prints to approximately 1,600 printers worldwide, serving 10,000 UNIX and Windows 95 users, some of whom are in mission-critical environments.
rdist

rdist is a tool which allows the mirroring of directories between servers. This can be done either by doing an exact mirror (including the deletion of files) or just adding to and replacing directories already present.

Using this tool, we could put together a directory structure on a master distribution server and have it mirrored throughout all the other servers automatically. This was yet another massive timesaver, as the number of servers we managed increased. We even included the /etc/passwd and /etc/group files among those files updated using rdist.

Setting up a new server became straightforward: install a vanilla Red Hat release, then rdist the print system on top.

New Web Stuff

I did another rework of the web pages. I added the ability to stop and start print queues as well as delete a print job and send a test page.

I discovered that while using SNMP, I could display an HP printer's front panel display, which greatly aided in fixing run-of-the-mill problems such as “Toner Out”.

The web interface became the preferred tool for diagnosing printer problems and was made available to everyone. It allowed users to fix many of their own printer problems and not have to call us.

Failover

As I mentioned before, loccodes provided a rudimentary failover procedure. However, it would have been an arduous task to update all those loccode and sgroup records to denote a new server in case of failure. Luckily, I have never had to use it.

I added a backup server field to the loccode and sgroup records and created a new SDDB table called pserver. In it was detailed, among other things, the state of the server (“up” or “down”). I changed mkprint so that when it created the configuration files, it would check the state of the primary server, as designated on each loccode and sgroup record. If a server was marked as “down”, mkprint would direct LPR to use the backup server instead.

Now, failover was simply a matter of marking the dead server as “down” in SDDB and re-running mkprint on all the servers.

A Back End for LPR

The only problem with the mkprint system was that any updates to SDDB records were not instantly reflected in the print servers. Could I make the various programs bypass their configuration files and read SDDB directly?

I pulled open the LPR code and again replaced the routines that read the /etc/printcap file. This time, however, they read an SDDB record and created an in-memory printcap entry accordingly, using the same algorithms that mkprint used. I then did the same thing with BOOTP. It no longer needed to read /etc/bootptab, but rather read its information straight from SDDB.

Now, whenever changes were made in SDDB, print queues were created instantly and the BOOTP server was immediately available to service the BOOTP request. Failover didn't even require mkprint to be run. Samba was the only program left that didn't read its information directly from SDDB.

Linux Takes Over

After an extreme power outage at the San Jose headquarters' data center and general lack of performance due to capacity problems, the visibility of the print system was raised. Suddenly, managers realized that without the print system, nothing got printed and production lines stopped. The order came down to “double the capacity of the print system immediately”. I was given an assistant, Ben Woodard (bwoodard@cisco.com) to provide a much needed extra pair of hands. Although originally employed by Cisco as an MS Windows support line technician, within a few months I had converted him to a die-hard Linux fan.

We installed ten new Linux servers in the San Jose headquarters' data center. Linux print servers were spread around the world:

  • 13 at the San Jose headquarters

  • 3 around Silicon Valley

  • 4 in Europe

  • 2 in RTP, North Carolina

  • 2 in Tokyo, Japan

  • 1 in Sydney, Australia

On August 1, 1997, we retired the SPARC 20s and the other Axil machines. We are now completely reliant on Linux servers—a single line of commercial code cannot be found. We have (or at least have access to) the source code for every program we are running.

The Future

We have recently started replacing the print function in the Cisco branch offices. Until now, they have used local NT servers (with their associated problems) to manage their printers. Although the branch offices only have 5% of our printers, they account for about 50% of our calls. We have (as of February 1998) deployed 30 Linux servers to these branch offices, with many more on the way—there are currently about 200 branch offices in all.

There is still plenty of work to do as we expand the system to meet the needs of our printing clients. We have the following goals in mind:

  • Improve SNMP features. Enable traps so the printers tell us when they have a problem.

  • Create print queues using Java. The creation of print queues still requires someone who is happy to log onto the system and run pradmin, the printer queue administration program. Why not allow anyone with a browser and the right authorization to create queues?

  • Replace the LPR program with LPRng or even an implementation of the Internet Printing Protocol (IPP). LPR is showing its age and does not provide many facilities (sending printing options such as duplex, for example).

  • Extract page counts from the printers, using SNMP. We should be able to schedule regular maintenance visits by engineers.

  • Implement a DHCP server. Many printers now support DHCP as well as BOOTP. A DHCP server would allow us to avoid allocating IP addresses to printers.

  • Work on SDDB in order to make it usable as a general purpose Network Directory Service. I also need a new name for it.

______________________

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix