An Introduction to Embedded Linux Development, Part 3
At the end of the second article in this series, which outlines a step-by-step process for embedded Linux development, we had the LBox up and running, ready to use for application development. On some occasions, however, the developer needs to modify the underlying system. Any Linux system can be considered to consist of three parts: the root filesystem, the kernel image and the bootloader (the BIOS plays this role in the desktop system). Typically, we leave the bootloader alone--at least we hope to--and modify the root filesystem, the kernel image or both.
With this need for modification in mind, Part 3 of this article series consists of four sections:
Memory organization and filesystem layout
Replacing the kernel and root filesystem
Replacing the JFFS2 filesystem
Replacing the bootloader
Some of these activities can be dangerous and may result in a non-functional system. Recovery from that state will be discussed in the fourth and final installment of this introductory sequence.
We continue to work with the SBC (single-board computer) we used in Parts 1 and 2 of this series, the LBox from Engineering Technologies Canada Ltd. Nevertheless, much of the material has broader applications and should be useful for a variety of target platforms.
To replace the root filesystem and/or kernel image intelligently, it is useful both to understand the boot-up process in a general way and to have a grasp of memory organization. Useful details can be obtained from the SBC vendor; if Linux is resident, the operating system itself can be a source of information. In particular, when connected to the LBox we found these Linux commands to be informative: dmesg, df, cat /proc/mtd, cat /proc/meminfo and cat /proc/mounts.
The memory and filesystem layout differ in their details, depending on the SBC. Using the aforementioned commands and information from the vendor, the following information emerges. Our LBox memory consists of 4MB of Flash memory and 8MB of SDRAM. At the start of the boot process, the RAM is not yet deployed and the Flash organization is as follows:
device is /dev/mtd0
space provided = 0x0000E000
location: 0x00000000 -> 0x0000E000
device is /dev/mtd1
space provided = 0x00002000
location: 0x0000E000 -> 0x00010000
Image (root filesystem and kernel image, both compressed)
device is /dev/mtd2
space provided = 0x00100000
location: 0x00010000 -> 0x00110000
JFSS2 (Journaling filesystem to be mounted during boot)
device is /dev/mtd3
space provided = 0x002F0000
location: 0x00110000 -> 0x00400000
At boot, the typical SBC with Linux resident has a bootloader that finds the compressed kernel image and root filesystem in Flash, decompresses them into RAM and transfers control to the decompressed kernel in RAM. The kernel then completes the initialization of the system.
The LBox provides a specific example of the foregoing. When it powers up, the bootloader, in Flash, begins executing. It decompresses the root filesystem and the kernel image and moves them into the 8MB of SDRAM. In addition, part of the SDRAM is used for the root filesystem, which is set up as a ROM filesystem. Then, a ramdisk, /dev/ram1, is mounted on /var. Finally, the JFFS2 filesystem, /dev/mtdblock3, is mounted on /etc/config. As a result, the root filesystem, originally a totally ROM filesystem, has been modified by the mount events to include read/write areas--a ramdisk and a JFFS2 area. Things to notice:
The ROM filesystem intrinsically is read-only and its contents will be reconstituted at the next boot sequence.
The ramdisk is volatile and its contents will be lost at power down.
The JFFS2 filesystem is on Flash and its contents will persist through power-down events, so that it emulates a hard drive.
The filesystem organization of a pristine system, after boot, looks like this:
root file system (romfs) in RAM, read-only
/dev/root mounted on /
size = 1113 kbytes
ramdisk (ext2 file system) in RAM, read/write
/dev/ram1 mounted on /var
size = 115 kbytes
emulated hard drive (JFFS2 filesystem) in Flash, read/write
/dev/mtdblock3 mounted on /etc/config
size = 3008 kbytes
Various reasons exist for changing the kernel and/or root filesystem. Compiling in new hardware support, updating existing driver modules or adding new capabilities to the root filesystem are examples.
As you may have noticed above, before boot the kernel image and root filesystem, both compressed, are on the same partition, /dev/mtd2. Before shipping, the vendor placed the kernel image and the romfs for the root filesystem into the same compressed file, image.gz, which then was flashed into the /dev/mtd2 partition in Flash memory. We retain this approach, so that updating either the kernel image or the root filesystem involves reflashing both.
Let's look at how these software components are put on the LBox. To update software, the LBox makes use of a uClinux utility called netflash. In order to use this utility, a kernel must be running on the LBox. Further, the workstation must have the tftp (trivial FTP) daemon running, because netflash relies on that. It is outside the scope of this article to cover the installation of tftp, but most distributions have some sort of package management tool that allows one to install tftp, which is part of the larger netkit package. As of this writing, the current version is netkit-tftp-0.17, which can be found here. In recent years, security concerns have led us to avoid FTP and its variants in favor of sftp. You also may need to change firewall rules or turn off the firewall on the workstation. Consequently, we strongly recommend prudent paranoia; that is, tftp should be used when you are networked to the LBox only, not to the Internet itself. (See the section called "NFS Mounting" in Part 2 of this series for more details.)
The netflash utility allows us to transfer a file from the host computer by way of the network connection and save it in the flash memory on the target device. The command is entered on the LBox. For example, to retrieve a synopsis of its parameters and options, enter netflash -h to get a help screen.
One option we use is the -b option, which prevents automatic rebooting after the flashing operation is complete; otherwise, the LBox reboots when netflash completes. The -b option therefore provides a layer of protection should we suddenly realize the software component we just flashed was the wrong one or one that doesn't work. We also note that the netflash utility complains if you try to load a file larger than the partition you are trying to flash. The failure messages are not always informative, however. For example, it merely may provide a usage synopsis when it fails for some underlying cause, even though the syntax was okay.
So, from where do we get the image.gz file? This is accomplished on the workstation, where we use the tool chains that come with LBox. Recall that these use uClinux for kernel and root filesystem configuration. The uClinux directory hierarchy descends from the uClinux-dist directory node we installed on the workstation in Part 2 of this series. We assume that is our working directory when giving relative pathnames below. As necessary, we also use absolute pathnames.
To change the kernel and/or root filesystem, we must run make menuconfig or make xconfig as root from the uClinux-dist directory. This command presents a high-level GUI from which we can choose "Target Platform Selection" to open a new lower-level GUI. The lower-level GUI then allows us to configure the kernel, the vendor settings or both. The GUI at this level is somewhat non-intuitive. In particular, after making a selection, the Next button does not become active. Instead, after making our selection, we must close the lower-level GUI, returning to the original high-level GUI. We then choose the Save and Exit button, whereupon we are presented with a new GUI for making kernel and/or vendor setting modifications.
We suggest that for the first time through, no changes be made. Then, after the make xconfig step has been completed, enter make dep and make all. These result in writing the new image.gz file to both the images subdirectory and the tftpboot directory. The latter is accessed by the LBox using netflash.
When we are sure that all has been done correctly, we can enter the following command on the LBox--for example, by way of minicom--to update the kernel image:
netflash -knrb /dev/mtd2 tftp_server /tftpboot/image.gz
where tftp_server is replaced with the IP address of your workstation.
Recall that the /dev/mtd2 device corresponds to the joint kernel/root filesystem partition. Also, as noted above, we use the -b option to prevent automatic reboot after completing the update of the flash memory.
If you have trouble using the netflash utility, here are some things to check:
Make sure the file you are flashing actually exists in /tftpboot. Then, confirm that the /tftpboot directory permissions are 777 and the image.gz file permissions are 666.
Check that the tftp daemon is running. On the workstation, enter the command netstat -a | grep tftp.
Ensure that the network interface between the LBox and host computer is working properly.
Ensure that no firewall rules are preventing the file transfer by tftp.
Of course, we can modify the JFFS2 filesystem while the LBox is running--after all, it acts as our hard drive. If we want to make a dramatic overhaul, however, we could reorganize and reflash the whole thing. Doing exactly that is the subject of this section. We again would use netflash to flash the whole /dev/mtd3 partition with a JFFS2 image. Doing so leaves the JFFS2 flash partition metadata out of sync with filesystem information--for example, inode data--that the kernel brought into memory earlier. Therefore, rebooting after the netflash operation seems okay is recommended.
A few considerations must be made when getting the user filesystem ready to netflash to the LBox. First, we need to construct a directory on the workstation that contains what ultimately will be the contents of the JFFS2 directory on the LBox. This directory already exists on the CD provided by the vendor, and we copied it over to the workstation earlier. Its name is etc_config/. We suggest that it be left as is for the trial run that follows.
Next, we use the following command to create an output file, calling it config.jffs2:
mkfs.jffs2 -b --pad=0x002f0000 --root=etc_config/ --output=config.jffs2
The --root option specifies that the etc_config/ directory is used to provide the intended contents for the JFFS2 filesystem. The --output option identifies the name, config.jffs2, for the file that will be created containing the desired JFFS2 filesystem. The --pad option pads the result out to the same size as the /dev/mtd3 partition on the flash memory. The -b option is needed to create the filesystem as big-endian, appropriate for the embedded Motorola architecture.
After the filesystem is ready, we move it into the /tftpboot directory on the host computer. We then run the following command from the LBox, by way of minicom, to flash the filesystem:
netflash -knrb /dev/mtd3 tftp_server /tftpboot/config.jffs2
where tftp_server is replaced by the IP address of your workstation.
Notice that the device is /dev/mtd3 and that this corresponds to the memory location for the JFFS2 filesystem. After rebooting the LBox you should see the new filesystem residing at /etc/config on the LBox.
The bootloader is the software component least likely to require an update. However, there are times when you may need to do this. Perhaps you need to compile in new features, or maybe a newer version of the bootloader is needed. As with the kernel/root filesystem update, updating the bootloader also can be risky. Now, with the new bootloader binary in the tftp directory, we can update the LBox by executing the following command from the LBox.
netflash -bknr /dev/mtd0 tftp_server /tftpboot/colilo.bin
where tftp_server again is replaced by the IP address of your workstation.
In this case, we put the binary file into the bootloader partition, which is /dev/mtd0. We again used the -b option to give us one more chance to recover should we suddenly realize that the colilo.bin file has a problem. That is, we still could repeat the above steps with the correct file. If everything goes well, we can reboot the LBox and ultimately be presented with the command prompt.
Several of the netflash/reboot iterations discussed in the current article can leave the LBox in a non-functional state if the transferred files are not correct. In particular, if the Linux image is bad, netflash will not be available to access the workstation's tftp server; if the colilo binary is bad, we can't even start the boot process. To recover, we can use a Background Debug Mode (BDM) Coldfire cable to access the BDM of the Motorola Coldfire microcontroller. The next and final installment of this series will explain how to conduct such a recovery operation.
Richard Sevenich is a Professor of Computer Science at Eastern Washington University in Cheney, WA.
Ben Anderson is an upper division Computer Science major at Eastern Washington University. His current interests are hardware and embedded systems, particularly those which use Linux.