Large-Scale Linux Configuration Management
A number of components on each machine take the resources from the repository and implement the specified configuration in whatever way is appropriate for that particular platform. The components are currently implemented as shell scripts which take a standard set of method arguments, rather like the rc.d startup scripts under Red Hat Linux:
START: executed when the system boots.
STOP: executed when the system shuts down.
RUN: executed periodically (from cron).
A client-server program (om) also allows methods to be executed on demand on multiple remote machines. Components may have other arbitrary methods in addition to the standard ones.
Different types of components will perform different actions at different times. Typically, a daemon might be started at boot time, reloaded periodically, and stopped at shutdown. Some components however, might simply perform a reconfiguration at boot time, or start only in response to the RUN method (for example, a backup system).
Component scripts normally inherit a set of subroutines from a generic component. This provides default methods and various utility procedures for operations such as resource retrieval. This makes simple components easy to write, and scripts are frequently quite short.
A typical host runs 20 to 30 components, controlling subsystems such as web servers, printers, NIS services, NFS configuration and various other daemons. Two components are worth mentioning in more detail.
The boot component is the only one run directly from the system startup files. This uses resources to determine which other components to start. The set of services running on a particular machine is therefore controlled by the boot resources.
The update component normally runs nightly, as well as at boot time. This uses the extremely useful updaterpms program which compares the RPMs installed on a machine with those specified via the resources. RPMs are automatically installed or deleted to synchronise the state of the machine with the specification. This means that all machines in the same class are always guaranteed to have identical sets of up-to-date packages. Changing an inherited resource file will automatically reconfigure the RPMs carried by all machines in the class.
As much configuration as possible is performed dynamically by the various components. However, some configuration, such as disk partitioning, must be hard-wired at installation time. New machines are booted using an installation floppy, which mounts a root file system from the network, a CD or Zip drive. The boot process runs a special install component which determines all necessary install-time parameters by interpreting the machine's install resources. A very minimal template is installed on the new system and the update component is used to load the initial set of RPMs.
This supports completely unattended builds of new machines, as well as rebuilds of existing machines. If there is any doubt about the integrity of a system, it is normal for us to simply rebuild it from scratch.
The concept of an open, lightweight framework has been very important; many people have contributed components so that virtually everything which varies between our machines is now handled by LCFG. This has made the system very successful; however, much of the implementation is still based on technologies originally intended to be temporary. We are currently planning to expand the use of LCFG beyond our own department and this is motivating a redesign of some of the subsystems, although the basic architecture will remain the same:
We hope to implement a new syntax for specifying the resources, together with a special-purpose resource compiler.
We hope to replace the NIS distribution with something simpler which is available earlier in the boot sequence.
We would like to re-implement the components in Perl, using Perl inheritance to provide generic operations.
Other items on the wish list include caching support for portables and secure signing of resources.
Paul Anderson is a Senior Computing Officer with the Division of Informatics at Edinburgh University. He has been involved with UNIX systems administration for 15 years. Further information is available from www.dcs.ed.ac.uk/~paul, and comments by e-mail are welcome at firstname.lastname@example.org.
- NSA: Linux Journal is an "extremist forum" and its readers get flagged for extra surveillance
- Secret Agent Man
- Own Your DNS Data
- Smoothwall Express
- Returning Values from Bash Functions
- Bash Shell Script: Building a Better March Madness Bracket
- Simple Server Hardening
- Machine Learning Everywhere
- Tech Tip: Really Simple HTTP Server with Python
- Brent Laster's Professional Git (Wrox)
Pick up any e-commerce web or mobile app today, and you’ll be holding a mashup of interconnected applications and services from a variety of different providers. For instance, when you connect to Amazon’s e-commerce app, cookies, tags and pixels that are monitored by solutions like Exact Target, BazaarVoice, Bing, Shopzilla, Liveramp and Google Tag Manager track every action you take. You’re presented with special offers and coupons based on your viewing and buying patterns. If you find something you want for your birthday, a third party manages your wish list, which you can share through multiple social- media outlets or email to a friend. When you select something to buy, you find yourself presented with similar items as kind suggestions. And when you finally check out, you’re offered the ability to pay with promo codes, gifts cards, PayPal or a variety of credit cards.Get the Guide