Bare Metal Recovery

Most us don't take the time to plan for disaster recovery. One excuse is not wanting to figure out what to do. One excuse down—this article gives you the step-by-step.
Creating the Stage One Backup

Having made your production backups, what additional information do you need to back up in order to rebuild your system? You need to preserve your partition information so that you can rebuild them. This and other metadata are preserved in the script save.metadata (see Listing 1).

Listing 1. save.metadata

Unfortunately, fdisk does not yet export partition information in a manner that allows you to reimport it from a file. Since you have to rebuild your partitions by hand using fdisk, we will save it as a human-readable text file.

Fortunately, the price of hard drives is plummeting almost as fast as the public's trust in politicians after an election. So it is good that the hand-editing process allows the flexibility necessary to use a larger replacement drive.

The script saves the partition information in the file fdisk.hda in the root of the Zip disk. It is a good idea to print this file and your /etc/fstab. Then, you can work from hard copy while you restore the partition data. You can save a tree by toggling between two virtual consoles, running fdisk in one and catting /etc/fstab or /fdisk.hda as needed, but this strikes me as error prone.

You will also want to preserve files relevant to your restoration method. For example, if you use nfs to save your data, you will need to preserve hosts.allow, hosts.deny, exports, etc. Also, if you are using any network backed restoration process, such as Amanda or Quick Restore, you will need to preserve networking files like HOSTNAME, hosts, etc. and the relevant software tree.

The simplest way to handle these and similar questions is to preserve the entire /etc directory.

Preston cheats when he backs up his system. There is no way a 100MB Zip drive is going to hold a server installation of Red Hat 5.2. A 250MB Zip disk will hold a fresh server installation, but probably won't hold a production server. We have to be much more selective than simply preserving the whole kazoo. What files do we need?

  • The boot directory.

  • The /etc directory and subdirectories.

  • Directories needed at boot time.

  • Device files in /dev.

To determine the directories needed at boot, we look at the boot initialization file /etc/rc.sysinit. It sets its path like so:

PATH=/bin:/sbin:/usr/bin:/usr/sbin
export PATH
Trial and error indicated that we needed some other directories as well, such as /dev. In retrospect, of course we need /dev. In Linux, you can't do much without device files.

In reading the script save.metadata (see Listing 1), again note that we aren't necessarily saving files that are called with absolute paths. We may require several iterations of backup, test the bare metal restore, reinstall from CD and try again, before we have a working backup script. While I worked on this article, I made five such interations before I had a successful restoration. That is one reason why it is essential to use scripts whenever possible. Test thoroughly!

Booting tomsrtbt

The first thing to do before starting the restoration process is to verify that the hardware time is set correctly. Use the BIOS setup for this. How close to exact you have to set the time depends on your applications. For restoration, within a few minutes of exact time should be accurate enough. This will allow time-critical events to pick up where they left off when you finally launch the restored system.

Before booting tomsrtbt, make sure your Zip drive is placed on a parallel port, either /dev/lp0 or /dev/lp1. The startup software will load the parallel port Zip drive driver for you.

I have one of those ne2000 clone Ethernet cards in my test system. This, it turns out, gives the 3c59x driver in the tomsrtbt kernel fits. The workaround is to tell the kernel to ignore its address range. At the LILO prompt:

zImage reserve=0x300,32

The next step is to set the video mode. I usually like to see as much on the screen as I can. So when the option to select a video mode comes, I use mode 6, 80 columns by 60 lines. Your hardware may or may not be able to handle high resolutions like that, so experiment with it.

Once tomsrtbt has booted and you have a console, mount the Zip drive. It is probably a good idea to mount it read only:

# mount /dev/sda1 /mnt -o ro

Check to be sure it is there:

# ls -l /mnt
Then clean out the first two sectors of the hard drive:
# dd if=/dev/zero of=/dev/hda bs=512 count=2
This sets the master boot record (MBR) to all zeros. It wipes out all record of the partitions and any boot code, such as LILO.

Then, using the hard copy of fdisk.hda you made earlier, use fdisk to partition the hard drive. Make the first cylinder a primary partition. After that, you will build one or more extended partitions with one to four logical partitions. Primary and extended partitions are numbered one to four. Partitions five and up are logical partitions. Notice also that the extended partitions overlap the logical partitions they contain. The partition number of a partition is the last one or two numbers in the device name, so /dev/hda5 is partition number five on hda.

You can have more than one primary partition, but this is unwise. Each primary partition precludes an extended partition, which can be subdivided into logical partitions. Extended partitions are far more flexible:

# fdisk

Command (m for help): p

Disk /dev/hda: 64 heads, 63 sectors, 1023 cylinders
Units = cylinders of 4032 * 512 bytes

Device Boot     Start   End     Blocks  Id      System

Command (m for help):

We will make a new partition as partition 1:

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1023): 1
Last cylinder or +size or +sizeM or +sizeK (1-1023): 9
Now we will make the extended partition:
Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
e
Partition number (1-4): 2
First cylinder (1-1023): 10
Last cylinder or +size or +sizeM or +sizeK (10-1023): 1022
Note that the hard drive has 1,023 cylinders, and we only made the extended partition go to 1,022, wasting a cylinder. The reason is that we are duplicating the setup Red Hat gave us on the original installation.

Now for a logical partition:

Command (m for help): n
Command action
   l   Logical (5 or over)
   p   primary partition (1-4)
l
First cylinder (10-1022): 10"
Last cylinder or +size or +sizeM or +sizeK (1-1022): 368

And so on for each partition that your fdisk.hda file indicates you should have.

Set the file system ID on your swap partition(s) to Linux Swap (82) using the t command.

Don't forget to use the a command to set a partition active, or bootable. In a Linux-only installation, this is usually the first partition, but can be another one. It will be the partition that mounts as /boot (see your printout of fstab):

Command (m for help): a
Partition number (1-9): 1

Then verify your work:

Command (m for help): v
372 unallocated sectors

Command (m for help): p

Disk /dev/hda: 64 heads, 63 sectors, 1022 cylinders
Units = cylinders of 4032 * 512 bytes

   Device Boot  Start   End     Blocks  Id      System
/dev/hda1   *   1       9       18112+  83      Linux native
/dev/hda2       10      1022    2042208 5       Extended
/dev/hda5       10      368     723712+ 83      Linux native
/dev/hda6       369     727     723712+ 83      Linux native
/dev/hda7       728     858     264064+ 83      Linux native
/dev/hda8       859     989     264064+ 83      Linux native
/dev/hda9       990     1022    66496+  82      Linux swap

Command (m for help):
Check this listing against your printed copy. Note that the one extended partition overlaps the five Linux partitions.

Finally, we use the w command to write the partition table and exit:

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
 hda: hda1 hda2 < hda5 hda6 hda7 hda8 hda9 >
 hda: hda1 hda2 < hda5 hda6 hda7 hda8 hda9 >
Syncing disks.

WARNING: If you have created or modified any DOS 6.x partitions, please see
the fdisk manual page for additional information.

Then make ext2 file systems on each partition you will be using as an ext2 partition. These will be the primary and logical partitions that you did not change to swap partitions. Don't do this to your extended partitions!

mke2fs /dev/hda1
mke2fs /dev/hda5
 ...
For example,
# mke2fs /dev/hda1
mke2fs 1.10, 24-April-97 for EXT2 FS 0.5b, 95/08/09
Linux ext2 file system format
File-system label=
4536 inodes, 18112 blocks
905 blocks (5.00%) reserved for the super user
First data block=1
Block size=1024 (log=0)
Fragment size=1024 (log=0)
3 block groups
8192 blocks per group, 8192 fragments per group
1512 inodes per group
Superblock backups stored on blocks:
        8193, 16385

Writing inode tables: done
Writing superblocks and file system accounting information: done
And so on for each primary or logical partition that is not a swap partition.

Now set up the swap partition.

# mkswap /dev/hda9
Setting up swap space, size = 68087808 bytes

Then, we have to manually build and mount a partition to each directory. Since what is now /target will eventually become /, we mount what will be / to /target:

# mkdir /target
# mount /dev/hda8 /target
Next, we build the directories we need in /target, and mount to them, like so for each directory:
# mkdir /target/boot
# mount /dev/hda1 /target/boot
You can determine which directories and mountspace points to build and mount from /mnt/etc/fstab. Fortunately, umask is already set correctly for almost all the directories we need to build.

And so on for each partition. Referring to /mnt/ls.root.txt, make sure you also set the proper modes for the directories you are building.

Do not try to mount the extended or swap partitions, though. You don't need to, and it won't do you any good, anyway.

To check your progress, use the command mount with no parameters.

# mount
/dev/ram0 on / type minix (rw)
none on /proc type proc (rw)
/dev/ram1 on /usr type minix (rw)
/dev/ram3 on /tmp type minix (rw)
/dev/sda1 on /mnt type ext2 (rw)
/dev/hda8 on /target type ext2 (rw)
/dev/hda1 on /target/boot type ext2 (rw)
/dev/hda6 on /target/home type ext2 (rw)
/dev/hda5 on /target/usr type ext2 (rw)
/dev/hda7 on /target/var type ext2 (rw)

Once you have created all your directories and mounted partitions to them, you can run the script /mnt/root.bin/restore.metadata (see Listing 2). This will restore the contents of the Zip drive to the hard drive.

Listing 2. Restore script! Zip to HD

You should see a directory of the Zip disk's root directory, then a list of the archive files as they are restored. tar on tomsrtbt will tell you that tar's block size is 20, and that's fine. You can ignore it. Be sure that LILO prints out its results:

Added linux *

That will be followed by the output from a df -m command.

If you normally boot directly to X, that could cause problems. To be safe, change your boot run level temporarily. In /etc/inittab, find the line that looks like this:

id:5:initdefault:

and change it to this:

id:3:initdefault:
Now, you can gracefully reboot. Remove the tomsrtbt floppy from your floppy drive if you haven't already done so, and give the computer the three-fingered salute, or its equivalent, “shutdown -r now”. The computer will shut down and reboot.

______________________

White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState