Supporting Multiple Kernel Versions

Expect scripts to help you support multiple versions of the kernel across different platforms.

I work in the Atlas experiment at CERN. Many of the groups in Atlas are beginning to turn to Linux as the operating system for the next generation of particle-physics experiments. Among the teams working on the data-acquisition, there is often a need for specific versions of the Linux kernel. Typically, the teams are using special PC cards, such as ATM cards, where the drivers may not yet be available for all kernel versions, or for which patches must first be applied to the kernel. Both SMP and uniprocessor machines are used, and team members want the same kernel with the same patches for both flavors. They also wish to share software and meaningfully compare results.

This article describes how my team supports this need. I will assume you are familiar with the basic process of configuring and compiling a kernel.

We needed an environment in which a range of kernels could be configured, built, packaged for distribution and later installed in a coherent and consistent manner. The result is our “kernel repository”, containing tar files of the source code for several kernel versions with different patches applied, tar files of the compiled kernels, and a set of scripts used to compile and install the kernels. We wanted to ensure that the configure and build steps were fully recorded and any kernel configuration could be reproduced at a later date, even if the details of the build environment had changed. Also, we wanted to keep up with newer kernel versions (at the time, 2.2.0 was about to be released), so we tried to make it easy to add new versions as they came out. We also wanted the distributions to be easy to install, so that people could use them without knowing the details.

Availability

Our kernel repository and all its associated tools are accessible on the WWW at www.cern.ch/Atlas/project/kernels/www/kernels.html. The kernels are available as separate distributions of precompiled binaries and source code. A technical note is included, which goes into greater detail on some points and helped form the basis of this article.

Preparing the Source Distributions

The original kernel sources were downloaded from the Web. Each kernel unpacks into a single subdirectory—/linux. Since users may want several kernel source trees available at the same time, we rename this directory to /linux-kernel_version_number.orig and repack the tree using tar and bzip2. To give a clear example, if I had downloaded the tar file for version 2.2.0 of the kernel, I would repack it with these commands:

cat linux-2.2.0.tar.bz2 | bzip2 -d | tar xf -
mv linux linux-2.2.0.orig
tar cf - linux-2.2.0.orig | bzip2 >linux-2.2.0.orig.tar.bz2

The kernel repository includes scripts which will fetch and repack kernels for you.

Whenever any patches were applied, the corresponding source tree was renamed to reflect the patch and sometimes the version of the patch. For example, 2.0.36 with the “bigphysarea” patch is packed as linux-2.0.36.bphys.tar.bz2.

Configuring the Kernels

To configure the kernels, I wrote an Expect script called KernelConfig.exp. Expect is a tool for automating interactive processes (see “Automating Tasks with Expect” by Vinnie Saladino, Linux Journal, October 1998), and it is ideally suited to this task. KernelConfig.exp runs make config at the top of the kernel source tree and answers the questions for you. The beauty of controlling the configuration by an Expect script is that it is insensitive to the kernel version used with it. This script should be able to configure any Linux kernel version. I have run it on all stable kernels from 2.0.33 to 2.2.7 and on a number of the 2.1-series kernels. While it may not give an optimal configuration for any kernel (whatever that might mean), it does provide consistent and reproducible configurations.

Some configuration options are hardwired in the script, such as:

  • Support for EXT2 and Minix file systems for compatibility with the majority of the available rescue disks.

  • Support for RAM disk and initial RAM disks. This is essential because the kernel uses modules for most of its functionality and needs the initial RAM disk support to boot on the widest possible range of hardware.

  • Compile the kernel for a Pentium-class processor—we have no 386 or 486 machines.

  • Support for NFS and for root-on-NFS. Eventually, booting over the network might be useful, in which case this option is mandatory.

  • Enable the “experimental code” option. This might seem dangerous, but it is needed to include the ATM code.

  • Module-versioning is disabled. While this means one less safety net for the user, it does make life easier if we have to move modules around.

  • Support for all classes of network cards, SCSI cards, sound cards and other classes of hardware is enabled. The individual card support is compiled as modules.

Some hardware support is plainly excluded, even though it is available as modules, because at least one of the kernels in the range we are using does not compile. Such problems are normally caught and fixed rapidly, but if the software is not important to us, I leave it disabled for consistency. This happens for a few sound cards, the infrared devices (broken in 2.2.6) and at least one ATM card we don't use. This list may grow as more kernels are produced, so you should look at the configuration log or the script to see what it does. For all other options, the script will select to compile the code as a module if possible; otherwise, it will use the default offered by the configuration. The output of the configuration is stored in the KernelConfig.log file in the current directory. If this file already exists, the output will be appended to it, not written over existing information.

Two words of caution are necessary. The script reacts to the defaults and will therefore react differently if the default changes. For some code, the default changes over time. Something available only as built into or excluded from the kernel in the 2.0.x series might be available as a module in the 2.2.x series. In this case, it will be built in or excluded from the 2.0.x series, according to the default, but will certainly be built as a module for the 2.2.x series. If you need that feature at boot time, be warned.

Secondly, the defaults are taken from the .config file if it exists, or from the file arch/i386/defconfig if the sources have never been configured. If you configure the kernel by hand and set some options before running KernelConfig.exp, it will accept those settings as default. For truly consistent results, run make mrproper before running KernelConfig.exp.

SMP support is tricky. In the 2.0.x series, SMP support had to be enabled by editing the Makefile or by building the kernel with the command make SMP=1 bzImage or a similar one. In the 2.2.x series, SMP support is a configuration-time option, and the Makefile no longer needs to be changed. KernelConfig.exp enables or disables SMP support, depending on the value of a user-defined environment variable SMP_SUPPORT. If this variable is not defined or is empty, the script will not enable SMP support. If it is non-empty, the script will enable SMP support in the 2.2.x series.

This is not enough for the 2.0.x series, where the value of the make-macro SMP must be true when the kernel is compiled, not when it is configured. I get around this by defining SMP_SUPPORT to have the value SMP=1. I can then run KernelConfig to configure the kernel, and make $SMP_SUPPORT bzImage afterwards. For the 2.0.x series kernels, the value of SMP_SUPPORT ensures that the kernel is built with SMP enabled at compilation time. For the 2.2.x series, the very fact that the variable is defined causes KernelConfig.exp to enable SMP support at configuration time. This gives a consistent approach to SMP for both the 2.0.x and 2.2.x series kernels.

______________________

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix