Easy Backup and Restore
Until recently the extent of my backup efforts was to take the occasional CD copy of my home directory and keep copies of important files somewhere else, usually on another disk partition or a floppy disk. All of this changed with the need to run some Windows legacy applications. The only machine really suitable for this work was my main workstation, a 1.2GHz Athlon machine, multibooted with four distributions. I decided to free up the first primary partition, which held Mandrake 9.0, and set up a Windows partition.
I freed up the first primary partition by transferring its contents to the seventh partition, overwriting an expendable Vector Linux 3.0 Distribution. To be totally safe, I booted into Debian 3.0 and mounted both partitions to individual mount points in /mnt. Then, as root, I used tar and a pipe to copy everything, including all links and permissions, from the source partition to the target partition. A few minutes later, after changing my GRUB boot menu, I was able to boot into Mandrake 9.0 Linux in the seventh partition and verify that everything worked as expected.
At this point, one normally would DOS-format the now free first partition and install Windows. I began to feel a little uneasy, however. Windows could format the whole darn drive or some other similar screwup could happen, in which case I would be in the position of having to fdisk the partitions and reinstall everything from scratch. The original disks, of course, would have all the applications except for those extra packages installed by me, but all custom configurations would be lost.
The machine now was running Mandrake 9.0, Debian 3.0 and Slackware 8.1. Of these, only losing my Slackware install would cause me grief, as it has been running like a top, boots to KDE 3.0 in less than 30 seconds--including my sign on--and is absolutely rock-solid stable. It also has the CUPS printing system set up perfectly for all of my printers on the LAN. So I must retain this setup at all costs. The solution, of course, is to back up fully everything from the Slackware install.
At this point, the desire to have a simple, easy and foolproof backup and recovery method took hold.
If you are a home or SOHO Linux user, I suggest that your backup and recovery system should:
Require no equipment or software other than what you already have
Be cost effective in backup media
Be easy to use regularly, or it will not be used at all
Be easy to verify, or it may be useless when the time comes
Require only the media and a working machine, in the hardware sense
Require only minimal knowledge of the recovery process when the crunch comes
A quick review of past Linux Gazette articles and a search of the Web turn up hundreds of backup solutions. Many are aimed specifically at the backup function; others are aimed at the repair and system recovery part of the overall effort to get back to some predefined state. Virtually none are customized for your system or your specific requirements, so why not roll your own solution? That is what we do here.
Most home or SOHO users do not have a tape drive system and are unlikely to purchase one for the sole purpose of backup, given that the cost of the tape system and software probably exceeds that of the computer itself. This essentially narrows our options to backup to removable disk, backup to the same or another hard drive, backup to a CD or backup over a network to some other hard drive. This last option essentially is a more complicated backup to local hard drive option, except there is zero chance of it being lost when your system goes down. So let us look at these options:
Floppy: Good for incremental backups on a daily basis and perhaps the best solution for saving work as it progresses, but useless for system-wide restoration. The LS120 Disk and the Zip disk are not large enough or common enough to be considered for the sort of simple but complete backup considered here.
Hard Drive: One can back up to a separate partition on the same drive, which is of little use if that drive fails, or one can back up to another hard drive in the same computer. This is good, except there is a fair chance that a power supply failure or nearby lightning strike could fry both drives--or somebody could steal the computer--leaving nothing to restore.
Network Filesystem Transfer: This is a good solution to back up and restore files for someone interested enough to correctly install it. however, it does nothing for the process of getting the system up again to the point where one can restore the files. In short, it's too complicated for most people to institute.
CD-ROM: This is where things begin to look interesting. These days most Linux users have a CD burner, and the availability of cheap CD-RW disks means the cost of maintaining something akin to the traditional rotating backup system is definitely manageable. This is the one for us.
The most essential requirement is to have a working and reliable CD burner. Any current Linux distribution has the required tools, and to minimize media costs, about $4 supplies two good quality CD-RW disks. For daily backups, these will last for about five and a half years, and used weekly, they will last for eternity.
The scheme proposed here is to use the two CD-RW disks to take backups in rotation. In my actual implementation, I have color-coded the spine of the disk covers red and green, respectively, to aid in the correct rotation.
We also require the backup disk to self-boot into a minimal working Linux system. This is necessary to ensure that we can re-establish the master boot record (MBR) and the rest of the original partition information, if required. This rules out using a boot disk image, as commonly supplied with the majority of distributions. These supply only a boot method and a Linux kernel, and they usually boot straight to the partition they are customized to boot.
After a quick perusal of the Linux on self-boot CDs available, I settled on using the classic and well tried TomsRtBt disk in a 2.88MB image format. This is not an ISO image, but it is suitable for being the boot image of an ISO we will burn. It is also to be found at various other sources on the Web. I have used this self-boot in the floppy format, and it is good and quite complete. Note that it also includes a Toms FAQ.
In order to restore our working Linux system to a given state, we need to have records of all the current directory contents that are changing on a day-to-day basis or have changed as customizations since the initial install. This can be done laboriously by inspection and detailed lists, which minimizes what must be restored. Or, it can be accomplished easily by backing up the entire contents of these directories.
In my case, I have decided to back up the entire contents of */home, /etc, /usr/local, /opt, /var, /root and /boot* on the Slackware 8.1 partition:
/home holds all of the files for each user
/etc holds all of the configuration information
/usr/local normally holds any extra programs added since the install
/opt is commonly used by applications to install files
/var holds all data of a variable nature
/root belongs to the root user and has essential customizations
/boot has all the files for booting the system and boot .conf files
In addition to the contents of each of the identified directories above, there are some more important pieces of information that one wouldn't want to be without if a sudden failure to boot occurred. These include a binary copy of the MBR, a text list of the partition table and a copy of the fstab file, in case you have forgotten which partitions correspond to which filesystem. Optionally, you could make a copy of the current XF86Config file and/or the text output of commands such as lsdev and lspci for full system information.
Also, how are we going to structure all of this information to ensure it gets onto the CD in such a way as to be completely self-contained and usable for the task at hand? Here is what I did. First, I created a directory to hold all of the information to back up. As root, I issued /mkdir /tmp/backup/. Note here that I am using /tmp as repository for the constant part of the backup CD. This is safe in Slackware, but it might not be in other distributions. Choose a safe location, and one not itself backed up by the tar file.
Put into the backup directory a copy of the TomsRtBt .img file
cp ./tomsrtbt288.img /tmp/backup/tomsrtbt288.img
Here the .img file is in my home directory.
Put into the backup directory a copy of the master boot record:
dd if=/dev/hda bs=512 count=1 > /tmp/backup/MBR.bin
The MBR holds the first stage of the boot mechanism you employ. In my case, that's stage1 of GRUB, the GRand Unified Boot Loader, or LILO, s well as the partition information for the primary partitions. The extended partition information is held elsewhere on the disk and can be restored--if required--with the information you store from the fdisk command detailed next.
Put into the backup directory a list of the partition information:
fdisk -l > /tmp/backup/Partition_Table
This will be used to compare with a Toms listing of the partition table before any restoration takes place.
Put into the backup directory a copy of fstab, which defines the file system mount points. Be aware that any errors here will make the files and devices inaccessible later:
cp /etc/fstab /tmp/backup/fstab.bak
Optionally, copy any other information you wish to have available to you before you are able to boot into your newly restored Linux system. For easy accessibility, I keep a copy of XF86Config on the disk to ensure that I always can set up X the way I like, even if I'm installing a new system upgrade. I also keep a copy of menu.lst, as I use GRUB as my boot loader of choice:
cp /etc/X11/XF86Config /tmp/backup/XF86Config.bak ... cp /boot/grub/menu.lst /tmp/backup/menu.lst.bak
These files are added to every copy of the backup disk that is burned. The disks need to be changed only if one of the files changes, and then it should be copied over.
Now we need to ask: what do we need to do to create our self-booting backup disk? Here's the procedure I used:
Create a compressed tar file of chosen directories and add it to /tmp/backup.
Create a bootable ISO of the backup directory by using mkisofs.
Check the size of the ISO to be sure it will fit on your chosen CD-RW disk.
Burn to CD-RW using cdrecord.
At appropriate stages, echo messages to standard out, such as md5sums and so on.
Clean up the files no longer needed.
The script to accomplish all of these steps is shown below. Be sure to make it executable--chmod 755 ./backup--and copy it to somewhere in root's PATH; /usr/local/bin is a good place. Do the same for the next script as well.
#!/bin/bash # backup #------------------------------------------------------------------------------ # Script to enable easy backup of all important Linux files # and also creates a customized system repair disk. # Uses two CD-RW Disks labled "RED" and "GREEN to rotate backups #------------------------------------------------------------------------------ # The backup directory already contains files for boot and recovery. # One can add more - my Slackware 8.1 system backup is < 580MB. Backup_Dirs="/home /etc /usr/local /opt /var /root /boot" Backup_Dest_Dir=/tmp/backup Backup_Date=`date +%b%d%Y` Image_File=/tmp/backup.iso declare -i Size # Create tar file with todays Month Day Year prepended for easy identification tar cvzf $Backup_Dest_Dir/$Backup_Date.tar.gz $Backup_Dirs &> /dev/null # Start backup process to local CD-RW drive echo "Backing up $Backup_Dest_Dir to CD-RW Drive - $Backup_Date" echo "Creating ISO9660 file system image ($Image_File)." mkisofs -b toms288.img -c boot.cat -r \ -o $Image_File $Backup_Dest_Dir &> /dev/nul # Check size of directory to burn in MB Size=`du -m $Image_File | cut -c 1-3` if [ $Size -lt 650 ] then echo "Size of ISO Image $Size MB, OK to Burn" else echo "Size of ISO Backup Image too Large to burn" rm $Backup_Dest_Dir/$Backup_Date.tar.gz # Remove dated tar file rm $Image_File # ISO is overwritten next backup but cleanup anyway exit 1 fi # Burn the CD-RW Speed=4 # Use best speed for CD-RW disks on YOUR system echo "Burning the disk." # Set dev=x,x,x from cdrecord -scanbus cdrecord -v speed=$Speed blank=fast dev=1,0,0 $Image_File &> /dev/null Md5sum_Iso=`md5sum $Image_File` echo "The md5sum of the created ISO is $Md5sum_Iso" # Could TEST here using Md5sum_Iso to verify md5sums but problem is tricky. echo "To verify use script md5scd, this will produce the burned CD's md5sum" echo "run this as User with backup CD in drive to be used for recovery." echo "This verifies not only the md5sum but that disk will read OK when needed." # Remove image file and tar file echo "Removing $Image_File" rm $Image_File echo "Removing : $Backup_Dest_Dir/$Backup_Date.tar.gz" rm $Backup_Dest_Dir/$Backup_Date.tar.gz echo "END BACKUP $Backup_Date" echo "Be sure to place this backup in the RED CD case and previous CD in GREEN" echo "------------------------------------------------------------------------" exit 0
Once established, the backup process is simple. I usually back up every day. If I'm not doing much on the system, then I back up every week instead. At the start of every backup, I place the CD-ROM from the green CD case into the CD burner. In an xterm I su to root and issue this command:
nohup backup &> /tmp/backup.log &/
and then close the xterm and go to bed. The backup takes about 15 minutes to complete, so it can be done at any convenient time in a work day. When next at the computer, I cat /tmp/backup.log and check that the backup went well.
If I also want to verify the backup ISO, I note down the first and last four or five letters of the listed ISO md5sum. As my burner does not reliably read back the CD just written, I transfer the CD to my CD-ROM drive and verify that the md5sums are identical, using the script md5scd (see the listing below). If they are identical, I put that newly burned CD into the red case and the last burned CD into the green case, ready for the next backup cycle. If any possibility of confusion exists, one can always check the date of the tar file. Note that because of the burner not reliably reading the backup CD, I have not included an automatic check of the md5sums. I did this because failure to validate does not mean the CD is in error, only what was just read from the burner was in error. In fact, I have never experienced a md5sum compare failure when using my CD-ROM. I consider the MD5 checksum to be essential, as even a single-bit error conceivably could corrupt the entire compressed archive.
#!/bin/bash #------------------------------------------------------------------------ # md5scd ---- Data CD md5sum Verifier # Script to automate determining Md5sum for ISO9660 CDs # NOTE - This script assumes that correct md5sum is known and one wishes # to verify that a particular CD copy has been burnt correctly. # If working from a downloaded ISO image use "md5sum ISO" at command line #------------------------------------------------------------------------ # Requires standard tools found in all Linux Distributions # If script invoked as user, check all permissions, groups, etc. # Missing arguments? if [ $# -ne 2 ] then echo "Usage - md5scd mountpoint device, ex - md5scd /mnt/cdrom /dev/hdc" exit 1 else : OK have arguments fi # Loop over md5sum determination ...100 good copies for install-fest? again=yes while [ "$again" = yes ] do echo "Please insert CD at $1 and press ENTER when ready" read #Wait for insertion of disk mount $1 Block_Count=`df -k $1 | grep $1 | cut -c 25-30` #700Mb disk cannot exceed this ^^^^^ column limit in 1k blocks umount $1 Md5sum_Cd=`dd if=$2 count=$Block_Count bs=1024 | md5sum` echo "The md5sum of the CD at $1 is " $Md5sum_Cd echo echo -n "Verify another CD? [yes/no]" read again # Wait for "yes" -> repeat, anything else -> drop through' done exit 0
Before that eventuality occurs, one should make sure the backup disk will boot. Also, make sure you understand the limitations of tomsrtbt, and practice reading the various files placed on the backup disk, as the only editor available there is Vi. When you need it, the disk first has to be mounted:
mount -t iso9660 /dev/xxx /mnt
It is possible to unzip and untar the tarred file using TomsRtBt by first using gzip and then tar. However, it probably is better to first check that the partition table is correct by using fdisk -l and a reference to the stored table. If not, to restore the MBR use:
dd if=/mnt/MBR.bin of=/dev/hda bs=1 count=512
This restores the primary partitions and the bootloader. Then, use fdisk and the partition table file to restore the extended partition manually along with the logical partitions within it. This all requires nerve and practice! However, any changes can be abandoned if you're not sure or are only practicing.
Next, do a clean install to the proper partitions. Reboot to the point where one has a root console, mount the backup CD and execute
tar xvzf /Mount_Dir/Backup_Tar_Filename
This restores all of the previous directories to their correct places. It should leave you with an almost fully restored system.
Note that if the problem is only lost or corrupt files, one also can restore any of the saved directories at any time by extracting with, for example:
tar xvzf /Mount_Dir/Backup_Tar_Filename /home
The test of any system is, "Does it work?" I initially verified that the backup CD does boot into Toms wonderful Linux system, that all of the text material was readable and that fdisk -l did correspond to the stored version. I did not reinstate the MBR from the binary image file, but it is there if I ever need it.
The final test was to restore my Slackware 8.1 system in place of the original Mandrake 9.0 system, before installing Windows and perhaps having to restore. In brief:
I changed menu.lst to reflect the fact that we now would boot Slackware not Mandrake and formatted the partition, mke2fs -j /dev/hda1.
I rebooted with the Slackware install disk in the drive and the BIOS set to boot from CD-ROM. In 15 minutes, everything was installed.
I rebooted into the new system, and at a root console I mounted the last backup CD to /mnt and issued tar xvzf /mnt/last_dated_backup.tar.gz.
In another five minutes, this reinstalled all of the backed up partition contents. A reboot now brought me into a restored Slackware 8.1 with X set up and a KDE login. Because of not saving /dev, some of the device permissions had to be reset--sound and so on--but this was trivial. The whole process took less than half an hour, a far cry from doing a normal install and adding all of the missing favourite programs, followed by a lengthy reconfiguration.
Backup is easy to do and easy to keep doing. Now that the system is in use, I see that a number of small improvements could be made. The manual backup and verification commands could be made shell variables and invoked with a single word. Also, if the total file size becomes a factor, one could use the --exclude option of tar to not include large sections of invariant code in the tar file. Alternatively, one could use bzip2 compression. As it is now, complete directories from root are saved.
The urgent need for the Windows applications turned out to be not so urgent, but the occasion gave me the prod I needed to actually back up regularly. Perhaps my next project will be to install Wine and try to get those pesky applications to run within Linux, without the need to always reboot.
I would be interested in any comments for improvement. Indeed any feedback would be welcome, particularly if glaring flaws or omissions are evident. This scheme has been in use for only a short time, but so far I'm very satisfied. Finally, I encourage you also to institute regular backups. If you want a quick ready-made approach, try this one. A few changes to the scripts should have you backing up today and everyday after that.
Copyright (c) 2003, Alan Keates. Originally published in Linux Gazette issue 91. Copyright (c) 2003, Specialized Systems Consultants, Inc.
A retired control systems engineer, Alan spent much of his career designing and implementing computerized control and shutdown systems for Canada's CANDU Nuclear Reactors. A programmer for over 40 years and a Linux enthusiast since 1994, his first log entry shows 7.83 Bogomips on a 386 DX33 machine still running. Linux and golf are in first and second place among many other hobbies.