Reducing OS Boot Times for In-Car Computer Applications, Part III
This is the final article in a three-part series (see Resources) on reducing boot times for in-car computers. The impetus for the project has been my company, CarBot, where we're trying to get computer hardware to behave as it should in a car. In the previous two articles, I discussed a range of solutions for solving the problem of slow boots. In the months since those articles were written, we've thoroughly explored the possibility of using a BIOS replacement on the VIA Epia-M boards in an attempt to reduce boot times to the theoretical minimum. During this time, we've also discovered some hacks that can be used to minimize the effect of boot times on the user experience.
In our earlier articles, we compared the unnecessarily slow boot process to that of a car radio. The car radio boot times have climbed from nearly instant in the 1980s to several seconds today, but they still are rapid enough to be hardly noticeable. If you watch a modern radio right after you start the engine, you may see it do a small power-on self test, flash all the lights on the unit--much as dashboards in many cars do--and then power on the amplifier, producing sound within a second or two.
We decided to use the radio as our benchmark of rapid usability and appliance-like behavior. We attempted to minimize the following two variables: time from computer power-on to video and time from computer power-on to audio.
Our testing rationale was if we can minimize the time before the arrival of audio and video in the boot process, we later can optimize our software so that whatever the user was seeing and hearing last can be restored. This is what a radio does, this is what a television does, this is what a computer should do.
It takes users a couple of seconds to orient themselves to the task at hand, such as configuring a car computer. If the screen looked like it did the last time they shut it off, it at least would give the illusion of being on and would be a little less annoying. In the book The Humane Interface, Jef Raskin, father of the Macintosh computer, asserts that there are no real technical reasons why a computer should take any boot time at all.
Reducing boot times can be done by brute force or through finesse. The solutions we have found or been told about can be broken down into three basic categories:
Reduce the boot time to its theoretical minimum.
Distract the user while the computer is booting up.
Anticipate the user, and boot before the user has a chance to wait.
Long ago, I saw a comment on the Linux BIOS page that EPIA-M support existed but was unstable. That fact had not changed since July 2003. I e-mailed the maintainer of that branch, and he said "The fastest time that I have is 5s with some optimization of [the] init script."
Wow, five seconds. We wanted to get that.
Initially, I tried to find a commercial vendor to replace the BIOS. General Software was kind enough to send me a sample of its BIOS replacement for the EPIA-M, but the company did not yet have a similar version for the EPIA M2, another motherboard we use in our products. Also, the entry price was surprisingly consistent among the several vendors I asked--around $10,000 dollars. Recouping the effort of making a custom BIOS requires this sort of pricing, I suppose. Part of the problem was these BIOS implementations did all sorts of great stuff, and what we were in search of was a BIOS that did almost nothing.
So we set out to see how much time we could recapture by throwing away the BIOS and booting Linux ourselves. The idea was to throw away all the useless BIOS functionality, show video at the boot loader (that is, GRUB splash screens) and start playing audio as early as we possibly could in the boot sequence.
Our Linux engineer, James, was sent off with a pair of Epia-M boards, a bootable 256MB Flash drive, the first two boot articles and a pointer to the LinuxBIOS site. I initially thought that we could get enough Linux into the BIOS chip itself to do something useful, such as squeak and Flash the monitor. That would have satisfied our light and sound objective, but James discovered that we did not have enough space without going to the Flash drive. So the new goal was to reduce the BIOS boot as much as possible and then launch a stripped ZipSlack distribution on the compact Flash IDE drive.
The main benefit of LinuxBIOS for our application also was the main drawback. The primary development focuses on making clusters of supercomputers that need to be able to reboot quickly. These clusters were designed for network booting, however; we wanted to boot off a disk or a Flash device.
The EPIA-M boards, although quite popular in the hardware hacking scene are not popular as clustered supercomputing nodes. The EPIA is not the fastest motherboard, running between 600MHz and 1GHz with a comparatively feeble CPU.
As the LinuxBIOS project has progressed, some of the code to do things other than bootstrap supercomputer nodes has fallen out of sync with the documentation. Downloading and reading most of the LinuxBIOS mailing list archives helped to reconcile the discrepancies. For instance, for a while it seemed as though the Etherboot modules would be adaptable to booting off of other devices, such as the compact flash/IDE adapter combo we were using. Eventually, James switched to using the FILO bootloader.
The hardware aspects of flashing a BIOS were the most adventurous. The first few compiles of a new BIOS image naturally produce a machine unable to boot. The essential tools for BIOS programming work is two of the same motherboard and the correct chip tweezers.
Restoring a fried BIOS seems wrong when you're doing it. When you fry your BIOS in computer A, you boot computer B with the fixed BIOS and then carefully extract its chip with the computer still running. Then, you put the other BIOS chip in its place and burn it with a backup of the original BIOS. Then, you can put the chip back in the other computer and continue on your way.
Getting a usable BIOS was no small feat. Etherboot would fail in many charming ways, for any reason, such as not finding the Ethernet card, not finding the kernel on the disk and so on. After a bad flash, either there would be nothing at reboot or LinuxBIOS would fail to load the BIOS itself or FILO. After a dozen or so failed attempts, a bootable BIOS finally was flashed, and ZipSlack was booted successfully off the CF card.
The following several sections, written by James, detail what was discovered along the way.
- Tech Tip: Really Simple HTTP Server with Python
- My Childhood in a Cigar Box
- Papa's Got a Brand New NAS
- Applied Expert Systems, Inc.'s CleverView for TCP/IP on Linux
- Returning Values from Bash Functions
- Rogue Wave Software's TotalView for HPC and CodeDynamics
- Simplenote, Simply Awesome!
- Panther MPC, Inc.'s Panther Alpha
- NethServer: Linux without All That Linux Stuff