Easy Backup and Restore

 in
Are you still not doing backups on a regular basis--or at all? With this easy-to-follow procedure and ready-made scripts, you're out of excuses.

Until recently the extent of my backup efforts was to take the occasional CD copy of my home directory and keep copies of important files somewhere else, usually on another disk partition or a floppy disk. All of this changed with the need to run some Windows legacy applications. The only machine really suitable for this work was my main workstation, a 1.2GHz Athlon machine, multibooted with four distributions. I decided to free up the first primary partition, which held Mandrake 9.0, and set up a Windows partition.

I freed up the first primary partition by transferring its contents to the seventh partition, overwriting an expendable Vector Linux 3.0 Distribution. To be totally safe, I booted into Debian 3.0 and mounted both partitions to individual mount points in /mnt. Then, as root, I used tar and a pipe to copy everything, including all links and permissions, from the source partition to the target partition. A few minutes later, after changing my GRUB boot menu, I was able to boot into Mandrake 9.0 Linux in the seventh partition and verify that everything worked as expected.

At this point, one normally would DOS-format the now free first partition and install Windows. I began to feel a little uneasy, however. Windows could format the whole darn drive or some other similar screwup could happen, in which case I would be in the position of having to fdisk the partitions and reinstall everything from scratch. The original disks, of course, would have all the applications except for those extra packages installed by me, but all custom configurations would be lost.

The machine now was running Mandrake 9.0, Debian 3.0 and Slackware 8.1. Of these, only losing my Slackware install would cause me grief, as it has been running like a top, boots to KDE 3.0 in less than 30 seconds--including my sign on--and is absolutely rock-solid stable. It also has the CUPS printing system set up perfectly for all of my printers on the LAN. So I must retain this setup at all costs. The solution, of course, is to back up fully everything from the Slackware install.

At this point, the desire to have a simple, easy and foolproof backup and recovery method took hold.

What Do We Really Need for a Backup and Recovery System?

If you are a home or SOHO Linux user, I suggest that your backup and recovery system should:

  • Require no equipment or software other than what you already have

  • Be cost effective in backup media

  • Be easy to use regularly, or it will not be used at all

  • Be easy to verify, or it may be useless when the time comes

  • Require only the media and a working machine, in the hardware sense

  • Require only minimal knowledge of the recovery process when the crunch comes

A quick review of past Linux Gazette articles and a search of the Web turn up hundreds of backup solutions. Many are aimed specifically at the backup function; others are aimed at the repair and system recovery part of the overall effort to get back to some predefined state. Virtually none are customized for your system or your specific requirements, so why not roll your own solution? That is what we do here.

What Can We Use?

Most home or SOHO users do not have a tape drive system and are unlikely to purchase one for the sole purpose of backup, given that the cost of the tape system and software probably exceeds that of the computer itself. This essentially narrows our options to backup to removable disk, backup to the same or another hard drive, backup to a CD or backup over a network to some other hard drive. This last option essentially is a more complicated backup to local hard drive option, except there is zero chance of it being lost when your system goes down. So let us look at these options:

  • Floppy: Good for incremental backups on a daily basis and perhaps the best solution for saving work as it progresses, but useless for system-wide restoration. The LS120 Disk and the Zip disk are not large enough or common enough to be considered for the sort of simple but complete backup considered here.

  • Hard Drive: One can back up to a separate partition on the same drive, which is of little use if that drive fails, or one can back up to another hard drive in the same computer. This is good, except there is a fair chance that a power supply failure or nearby lightning strike could fry both drives--or somebody could steal the computer--leaving nothing to restore.

  • Network Filesystem Transfer: This is a good solution to back up and restore files for someone interested enough to correctly install it. however, it does nothing for the process of getting the system up again to the point where one can restore the files. In short, it's too complicated for most people to institute.

  • CD-ROM: This is where things begin to look interesting. These days most Linux users have a CD burner, and the availability of cheap CD-RW disks means the cost of maintaining something akin to the traditional rotating backup system is definitely manageable. This is the one for us.

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

backing up files in linux or ubuntu

Anonymous's picture

I am a new user in linux, have to take it for class. How do you back up files?

After restore linux won't boot

smoky's picture

I am trying to recover my SLES ENT Server 9 from a tar backup. I have successfully restored the MBR and created the paritions and restored the full system using tar. When I try to boot the system after a restart I get the following error:

graphics file (hd0,1) /Message" Missing, press any key to continue"

I hit a key to continue and get the grub boot menu. I try to boot to linux but I get the following error:

Error 15: File not Found

Any ideas I can't seem to find any information on how to go further.

Thanks

Hard disk

Ferienhaus Ostsee's picture

I think the new hard disks are the best way to recover your data..

Backup

Serg's picture

I think as alternative for backup it is possible also to use Flash Memory Cards.
10 Years average guaranteed term of a storage of the data.

Use Flash Memory Cards

Autobusy's picture

Nice work on top of that it’s really helpful..

re

katalog's picture

Shouldn’t a mistress city give some sort of seclusion and msytery to its visitor? If so how ‘bout Park City, UT? High up in the mountains but still a very engaging and inspiring city for creatives. Dear old Philly would never think to look there…

Thanks for this!

spielen's picture

Thanks for this!

more flexible way to check size of cd/dvd in the script

nicholas biondi's picture

Great script my-man!
One minor thing I might add though, the line:

Size=`du -m $Image_File | cut -c 1-3`

is better served with the -f flag:

Size=`du -m $Image_File | cut -f1`

This way if your dvd is let's say 4700 mb's it still works. -f1 picks up the first field instead of the -c which parses characters.

shwoot!
-nicholas biondi

My way

David Sutton's picture

I keep a second identical hard drive on a removable drawer. Once a week I install it in the PC and run G4U disk mirroring software from a bootable CD. For a local disk copy I do the command "copydisk wd0 wdx" (x is the ordinal number of the destination disk) and it takes about 15 minutes. Free, quick, simple, OS-independant, and backs up all partitions at once.

Advice for young players: G4U can be dangerous if you have multiple working hard drives. You must be VERY careful to get the drive names (wd0, wd1, wd2) correct : otherwise you may find yourself backing up over a working drive, and the moment you realise its happening, it's too late.

If I need to restore from a drive failure, I would have to physically swap the backup drive for the failed drive, changing jumpers to make it the master drive along the way. If you need to restore individual files, just plug in the backup drive, and copy your backup files over to the working drive.

G4U also provides for backing up to another drive across a network, which avoids moving the backup drive around. Personally I like using the removable disk drawer as it provides protection from theft and other calamities. USB connected external hard disk housings would also work well and be a tad easier.

DVD instead of CD?

Allen's picture

Please help me...

What if I want to use DVD instead of CD?

How do I change the script? Does cdrecord write to DVD as well as CD?

If not, what do I use?

Thanks and take care,

Allen

Use rdiff-backup and pyBackpack

Anirban's picture

Here is my setup:

  • Fedora core 4 on x86_64
  • 2x160GB on LVM makes primary storage
  • 1x200GB on LVM makes backup storage
  • rdiff-backup
  • pyBackpack
  • Samba 3.0.x providing primary storage as mount points to my PCs.

This setup works quite well. I can save data from my PCs into the network mounted drive on the linux server and all the data is transparently backed up. It takes incrementals and is able to restore history.

Free professional backup solution from Arkeia

Anonymous's picture

Arkeia offers a no-cost license (up to 50GB) of its Linux based backup software, allowing both backup to disk and to tapes. This solution also brings network backup of Windows, xBSDs, Mac...

Download available from arkeia.org

ZIP as an alternative

Richard's picture

I have been using "find" and "zip" in combination for backups for years. And my main motivation for using zip instead of tar/gzip is your observation that "even a single-bit error conceivably could corrupt the entire compressed archive". Tar/gzip archives are great for distributing source packages, where you can simply get a new copy if yours goes bad, but they're not as good for backups.

Bit errors in a zip archive typically will make only the one file unrecoverable. And note that bit errors may develop over time -- just because your archive passes the md5sum check today, it might not in a year. CDs have an expected shelf life of 100 years, but remember, that hard drive that failed had a MTBF of 100,000 hours.

True

Ashley Bowers's picture

Yea I have yet to recover a bad file yet!

mbr equivalent for lvm?

Chris's picture

What would be the equivalent to the 'dd' statement for storing the layout of an lvm setup? Is it stored i a regular file, or is there a chunk of disk that needs copying?
(I don't have access to a linux box at the moment to investigate the answer for myself.)

Rsnapshot

Anthony Ettinger's picture

How could you have not mentioned rsnapshot? ;-)

www.rsnapshot.org

Good tips on rolling your own

Bruce Martin's picture

Thanks for a fine article on how to roll your own solution.
I have been using Hugo Rabson's MONDO from www.mondorescue.org to create backup disks for years. I think Hugo's code started with what you are doing here but has expanded. It still retains the kind of flexability and ease of use for personal backups. It has the capability to create CD-R, CD-RW, DVD-RW, ISO images, backups to NFS volumes and even to tape.

A CD-RW is far too small.

Anonymous's picture

Im sure it's nice, and all, but what about people with 160GB drives?

My collection of digital photos I've taken takes about 10GB alone.

I just have to trust in XFS and Western Digital :(

Well, if you value your

Anonymous's picture

Well, if you value your data, you should be buying a backup solution as part of your computer investment. So, if you can spend extra on a 300G hard drive, you should probably instead buy 2 160G hard drives, for instance. The price vs. size vs. quantity thing isn't quite that simple, but you get the idea. Backup solutions are not an "extra" thing; they're part of your basic costs. If you can't afford a backup system, then you can't afford the computer at all.

But, at the very least, partition the drive, and backups to the other partition with rsync or something like dirvish, as mentioned elsewhere.

re... too small

Anonymous's picture

I have the same problem. I have about 2 GB of saved docs and email, 3 GB of digital pics and about 10 GB of music that I don't want to get lost. I have the cd's for most of the music (yes I do own it) but to get it back onto the hard disk I would have to rip for a week.

My solution:
1) Grab an old pc out of the garage (PPro 200MZ in this case)
2) Install 3 of the biggest disk drives I had laying around.
2) Install Linux (RedHat 7.3 with all the Fedora Lagacy patches)
3) Plug it into the network. ( We now have NAS)
4) Set up rsync to backup nightly. (rsync only sends files that have changed)
5) Set up Samba to allow the wife to save her "My Documents"

Scripts:

$ cat sunday_sync
#!/bin/bash
sun_sync_doc_with_ast
sun_sync_pic_with_ast
sync_dudley_with_ast
sync_mp3_with_ast

$ cat sun_sync_doc_with_ast
#!/bin/bash
cd /home/me/
rsync -avz ./Documents me@192.168.1.50:./Doc1

$ cat sync_mp3_with_ast
#!/bin/bash
cd /home/me/Music
rsync -avz ./mp3 me@192.168.1.50:./Music

(you get the idea... I rsync nightly to this machine. I also have more then one directory on the target machine for doc and pic ... ie. ./Doc1 ./Doc2 ./Doc3 .... and rotate through the week. )

Notes:

You will notice that I have several copies of ./Doc on the backup server but only one copy of ./Music . Music doesn't change that often and if I did lose it it would be more of a hassle then a disaster. (You get extra points for making sure that ./Doc1 is on a different hard disk then ./Doc2 is on a different hard disk then ./Doc3)

In order to make the rsync happen with out my having to imput passwords I created an ssh key with no password. This is a little bit of a security hole but since this is sitting in my home office and you would have to hack my main pc to access the key... oh well. Even if you did this key is only used for the backup machine.

I choose RedHat 7.3 for the following reasons.
1) I had the disks sitting in my drawer.
2) With the Fedora Lagacy project putting out security updates it is as secure as any other version of RedHat.
3) Version 7.3 runs great on this old Pentium Pro 200. 7.3 was the high point for RedHat. It was fast, complete, stable, and ran well. Every version since has gotten more and more bloated and slower and slower on anything except "state of the art" hardware. In my opinion they are close to if not already living in Microsoft Bloatsville. RedHat needs to force their developers use 500MZ Pentium II machines for their desktops.

if you want rsync web

Anonymous's picture

if you want rsync web interface or gui tools check this very useful

rsync web interface or gui tools

Have a look at dirvish.you

Anonymous's picture

Have a look at dirvish.
you can use it on your local computer and via the network. it also uses rsync for transferring the data, but you can keep multiple backups of your data wihout wasting space for common files. this is achieved by using hardlinks for common files. it also provides indexing and search functionality. I use it for several months on our servers and had no problems...

Yes. Something like Dirvish

Anonymous's picture

Yes. Something like Dirvish is ideal, I think. Optical media is far too unreliable for backups, imho. At best, they're useful for shipping data or transferring to a non-networked machine. I was using faubackup for a while, to transfer from one machine's HD to another's, but dirvish is similar, only faster; just got finished setting it up myself. It's a little more complex than faubackup (which would still be useful for very simple backups or backups to the same machine), but not too difficult to get your head around, and quite flexible and powerful too.

A CD-RW is far too small.

Tom Simonsen's picture

For large amounts of data, an external USB2 or firewire drive is the only practical way to go.

It should be possible to modify this script to use an external USB disk instead. For rotation and "security" use multiple directories or partitions.

I think you switched the bs

Anonymous's picture

I think you switched the bs and count #'s to restore the mbr.

D#ont use a CD-RW

Bob's picture

Correct, the only way is to use an external USB2, don't use a CD-RW, it's not suitable for data in large amounts.

External USb2 and firewir drive

Hilde's picture

Hi Bob, you are correct in your statement, but a firewire drive is also
possible.
Best regards Hilde from Germany

I read some articles and

Dobby's picture

I read some articles and learned a lot from your journal. Thank you and success for the future.
Best wishes
Dobby

Interesting articles

Carsten's picture

By reviewing your articles I can found out the same as Dobby(see his comment on September 9). I like your journal, especially the content but also the design.
Success in the future
Carsten

Thanks

Robert's picture

A helpful script for external USB!

By modifying the script use

Heinz's picture

By modifying the script use an external USB disk instead.
best regards
Heinz

my comment before

heiz's picture

sorry, my homepage behind my name heinz in not running. therefore
i try it again.
best regards
heinz

Easy backup

Diddi's picture

Easy backup and restore is a neverending story. I like the comments, helpful for my work.

White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState