Moving to SMP

For some, SMP, symmetric multi-processing, represents everyday work; for others, a hope for performance gains. Windows 9x users need not bother, since that platform supports only one CPU per host. Windows NT does not scale as well as other operating systems on multiple processors. The 2.0 Linux kernel series provided some SMP support, but the 2.2 series has much better support. Linux scales well, up to 16 CPUs. Beyond that, Linus Torvalds has not yet decided to commit, because the tradeoffs required to make Linux scale well to more would mean compromising performance for small systems. For large-scale SMP, Sun's Solaris does well at 64 CPUs. SGI and IBM also have large-scale SMP offerings. This article provides an introduction to running SMP under Linux on the x86.

For some time, I had wanted to experiment with SMP Linux. When the 2.2 kernel appeared and another hardware transaction in my home network left me with a 486 for my desktop machine (my wife got the K6-200 to go with her new keyboard, where the integrated pointer required the only PS/2 mouse port in the house), I decided to replace the 486 with something a bit different.

While looking longingly at multi-CPU Alpha systems and Intel P6 and Pentium II motherboards, I decided to build an SMP machine at a lower cost, but with new parts. Alpha and Ultra-SPARC systems did not have the low price I preferred to pay, and since I had a Baby AT case, Pentium II motherboards would not work with the hardware I could recycle. That left Baby AT-size P5 and P6 motherboards. I did not find any new P6 motherboards at appealing prices, but I did find a few P5 boards that would work.

After conducting more research, including reviewing past discussions on SMP at Slashdot, I chose the Tyan Tomcat IV 1564D, which sports the Intel 430HX chip set. This board can use one or two (preferably matched) Pentium processors from the P5-75 to the P5-233 MMX. It can also hold up to 512MB RAM, spread across eight 72-pin SIMM slots, using parity or non-parity memory devices. It has typical on-board I/O, including two IDE, two serial, one parallel, one PS/2 mouse port and USB. It can also use non-Intel CPUs, but will support only one, since Intel had de facto control of SMP in the x86 world until the AMD Athlon reached market, and the non-Intel offerings that support any SMP do not use the Intel signalling.

I purchased two P5-233 MMX CPUs and the motherboard from Motherboard Express. I added 128MB of fast-page parity memory from Crucial Technology.

For proper operation, SMP should combine with a thread-safe libc, such as glibc2. I run Debian GNU/Linux version 2.1, which has all libraries and utilities up to date for SMP. Debian's package set also includes libc5 libraries for software compiled to require libc5. After three years of near-continuous use, I find Debian the most pleasing Linux distribution. I have had no trouble performing upgrades and keeping current with updates. The package manager requires no manual downloads, and retrieves only the packages necessary for staying current. (My previous and parallel experience with Red Hat, through version 5.2, found no such facility built into or near RPM. If it exists, I missed it.)

Some drivers also require updates in order to perform correctly under an SMP kernel, since additional locking must occur to reduce contention for system resources. 4Front Technologies' OSS sound driver comes in uniprocessor and SMP varieties. PCMCIA Card Services may require recompilation. Most other drivers reside in the kernel source tree, so they should work with SMP after compiling a new kernel.

After replacing the 486 motherboard with the new Tyan unit, Linux booted straightaway. I already had the 2.2 kernel running, and so reconfigured it for SMP. (See smp.txt in the Documentation subdirectory of the Linux kernel source for more on how to perform this task.)

The first SMP kernel I compiled did not work correctly. From my reading of the documentation included with 4Front Technologies' OSS sound drivers and in the kernel itself, I realized the dependencies didn't get built correctly. I saved the .config file elsewhere, performed make mrproper to clean the kernel source tree, then restored the .config file. After performing make oldconfig, I built again and installed the SMP kernel. On the next boot, I saw additional startup messages to indicate that both CPUs had started running. The 2.2.7 kernel, in conjunction with the utilities shipped with Debian 2.1, report each process's CPU usage as a percentage of the total available. A process consuming all of one CPU will show 50% usage.