Easy Backup and Restore
Until recently the extent of my backup efforts was to take the occasional CD copy of my home directory and keep copies of important files somewhere else, usually on another disk partition or a floppy disk. All of this changed with the need to run some Windows legacy applications. The only machine really suitable for this work was my main workstation, a 1.2GHz Athlon machine, multibooted with four distributions. I decided to free up the first primary partition, which held Mandrake 9.0, and set up a Windows partition.
I freed up the first primary partition by transferring its contents to the seventh partition, overwriting an expendable Vector Linux 3.0 Distribution. To be totally safe, I booted into Debian 3.0 and mounted both partitions to individual mount points in /mnt. Then, as root, I used tar and a pipe to copy everything, including all links and permissions, from the source partition to the target partition. A few minutes later, after changing my GRUB boot menu, I was able to boot into Mandrake 9.0 Linux in the seventh partition and verify that everything worked as expected.
At this point, one normally would DOS-format the now free first partition and install Windows. I began to feel a little uneasy, however. Windows could format the whole darn drive or some other similar screwup could happen, in which case I would be in the position of having to fdisk the partitions and reinstall everything from scratch. The original disks, of course, would have all the applications except for those extra packages installed by me, but all custom configurations would be lost.
The machine now was running Mandrake 9.0, Debian 3.0 and Slackware 8.1. Of these, only losing my Slackware install would cause me grief, as it has been running like a top, boots to KDE 3.0 in less than 30 seconds--including my sign on--and is absolutely rock-solid stable. It also has the CUPS printing system set up perfectly for all of my printers on the LAN. So I must retain this setup at all costs. The solution, of course, is to back up fully everything from the Slackware install.
At this point, the desire to have a simple, easy and foolproof backup and recovery method took hold.
If you are a home or SOHO Linux user, I suggest that your backup and recovery system should:
Require no equipment or software other than what you already have
Be cost effective in backup media
Be easy to use regularly, or it will not be used at all
Be easy to verify, or it may be useless when the time comes
Require only the media and a working machine, in the hardware sense
Require only minimal knowledge of the recovery process when the crunch comes
A quick review of past Linux Gazette articles and a search of the Web turn up hundreds of backup solutions. Many are aimed specifically at the backup function; others are aimed at the repair and system recovery part of the overall effort to get back to some predefined state. Virtually none are customized for your system or your specific requirements, so why not roll your own solution? That is what we do here.
Most home or SOHO users do not have a tape drive system and are unlikely to purchase one for the sole purpose of backup, given that the cost of the tape system and software probably exceeds that of the computer itself. This essentially narrows our options to backup to removable disk, backup to the same or another hard drive, backup to a CD or backup over a network to some other hard drive. This last option essentially is a more complicated backup to local hard drive option, except there is zero chance of it being lost when your system goes down. So let us look at these options:
Floppy: Good for incremental backups on a daily basis and perhaps the best solution for saving work as it progresses, but useless for system-wide restoration. The LS120 Disk and the Zip disk are not large enough or common enough to be considered for the sort of simple but complete backup considered here.
Hard Drive: One can back up to a separate partition on the same drive, which is of little use if that drive fails, or one can back up to another hard drive in the same computer. This is good, except there is a fair chance that a power supply failure or nearby lightning strike could fry both drives--or somebody could steal the computer--leaving nothing to restore.
Network Filesystem Transfer: This is a good solution to back up and restore files for someone interested enough to correctly install it. however, it does nothing for the process of getting the system up again to the point where one can restore the files. In short, it's too complicated for most people to institute.
CD-ROM: This is where things begin to look interesting. These days most Linux users have a CD burner, and the availability of cheap CD-RW disks means the cost of maintaining something akin to the traditional rotating backup system is definitely manageable. This is the one for us.
Today’s modular x86 servers are compute-centric, designed as a least common denominator to support a wide range of IT workloads. Those generic, virtualized IT workloads have much different resource optimization requirements than hyperscale and cloud applications. They have resulted in a “one size fits all” enterprise IT architecture that is not optimized for a specific set of IT workloads, and especially not emerging hyperscale workloads, such as web applications, big data, and object storage. In this report, you will learn how shifting the focus from traditional compute-centric IT architectures to an innovative disaggregated fabric-based architecture can optimize and scale your data center.
Sponsored by AMD
Built-in forensics, incident response, and security with Red Hat Enterprise Linux 6
Every security policy provides guidance and requirements for ensuring adequate protection of information and data, as well as high-level technical and administrative security requirements for a system in a given environment. Traditionally, providing security for a system focuses on the confidentiality of the information on it. However, protecting the data integrity and system and data availability is just as important. For example, when processing United States intelligence information, there are three attributes that require protection: confidentiality, integrity, and availability.
Learn more about catching the bad guy in this free white paper.
Sponsored by DLT Solutions
| Using Salt Stack and Vagrant for Drupal Development | May 20, 2013 |
| Making Linux and Android Get Along (It's Not as Hard as It Sounds) | May 16, 2013 |
| Drupal Is a Framework: Why Everyone Needs to Understand This | May 15, 2013 |
| Home, My Backup Data Center | May 13, 2013 |
| Non-Linux FOSS: Seashore | May 10, 2013 |
| Trying to Tame the Tablet | May 08, 2013 |
- Making Linux and Android Get Along (It's Not as Hard as It Sounds)
- RSS Feeds
- New Products
- Using Salt Stack and Vagrant for Drupal Development
- Drupal Is a Framework: Why Everyone Needs to Understand This
- A Topic for Discussion - Open Source Feature-Richness?
- Home, My Backup Data Center
- Validate an E-Mail Address with PHP, the Right Way
- New Products
- Readers' Choice Awards
- Ahh, the Koolaid.
4 hours 27 min ago - git-annex assistant
10 hours 26 min ago - direct cable connection
10 hours 49 min ago - Agreed on AirDroid. With my
10 hours 59 min ago - I just learned this
11 hours 3 min ago - enterprise
11 hours 33 min ago - not living upto the mobile revolution
14 hours 25 min ago - Deceptive Advertising and
15 hours 50 sec ago - Let\'s declare that you have
15 hours 1 min ago - Alterations in Contest Due
15 hours 2 min ago
Enter to Win an Adafruit Prototyping Pi Plate Kit for Raspberry Pi

It's Raspberry Pi month at Linux Journal. Each week in May, Adafruit will be giving away a Pi-related prize to a lucky, randomly drawn LJ reader. Winners will be announced weekly.
Fill out the fields below to enter to win this week's prize-- a Prototyping Pi Plate Kit for Raspberry Pi.
Congratulations to our winners so far:
- 5-8-13, Pi Starter Pack: Jack Davis
- 5-15-13, Pi Model B 512MB RAM: Patrick Dunn
- Next winner announced on 5-21-13!
Free Webinar: Linux Backup and Recovery
Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.
In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.



Comments
backing up files in linux or ubuntu
I am a new user in linux, have to take it for class. How do you back up files?
After restore linux won't boot
I am trying to recover my SLES ENT Server 9 from a tar backup. I have successfully restored the MBR and created the paritions and restored the full system using tar. When I try to boot the system after a restart I get the following error:
graphics file (hd0,1) /Message" Missing, press any key to continue"
I hit a key to continue and get the grub boot menu. I try to boot to linux but I get the following error:
Error 15: File not Found
Any ideas I can't seem to find any information on how to go further.
Thanks
Hard disk
I think the new hard disks are the best way to recover your data..
Backup
I think as alternative for backup it is possible also to use Flash Memory Cards.
10 Years average guaranteed term of a storage of the data.
Use Flash Memory Cards
Nice work on top of that it’s really helpful..
re
Shouldn’t a mistress city give some sort of seclusion and msytery to its visitor? If so how ‘bout Park City, UT? High up in the mountains but still a very engaging and inspiring city for creatives. Dear old Philly would never think to look there…
Thanks for this!
Thanks for this!
more flexible way to check size of cd/dvd in the script
Great script my-man!
One minor thing I might add though, the line:
Size=`du -m $Image_File | cut -c 1-3`
is better served with the -f flag:
Size=`du -m $Image_File | cut -f1`
This way if your dvd is let's say 4700 mb's it still works. -f1 picks up the first field instead of the -c which parses characters.
shwoot!
-nicholas biondi
My way
I keep a second identical hard drive on a removable drawer. Once a week I install it in the PC and run G4U disk mirroring software from a bootable CD. For a local disk copy I do the command "copydisk wd0 wdx" (x is the ordinal number of the destination disk) and it takes about 15 minutes. Free, quick, simple, OS-independant, and backs up all partitions at once.
Advice for young players: G4U can be dangerous if you have multiple working hard drives. You must be VERY careful to get the drive names (wd0, wd1, wd2) correct : otherwise you may find yourself backing up over a working drive, and the moment you realise its happening, it's too late.
If I need to restore from a drive failure, I would have to physically swap the backup drive for the failed drive, changing jumpers to make it the master drive along the way. If you need to restore individual files, just plug in the backup drive, and copy your backup files over to the working drive.
G4U also provides for backing up to another drive across a network, which avoids moving the backup drive around. Personally I like using the removable disk drawer as it provides protection from theft and other calamities. USB connected external hard disk housings would also work well and be a tad easier.
DVD instead of CD?
Please help me...
What if I want to use DVD instead of CD?
How do I change the script? Does cdrecord write to DVD as well as CD?
If not, what do I use?
Thanks and take care,
Allen
Use rdiff-backup and pyBackpack
Here is my setup:
This setup works quite well. I can save data from my PCs into the network mounted drive on the linux server and all the data is transparently backed up. It takes incrementals and is able to restore history.
Free professional backup solution from Arkeia
Arkeia offers a no-cost license (up to 50GB) of its Linux based backup software, allowing both backup to disk and to tapes. This solution also brings network backup of Windows, xBSDs, Mac...
Download available from arkeia.org
ZIP as an alternative
I have been using "find" and "zip" in combination for backups for years. And my main motivation for using zip instead of tar/gzip is your observation that "even a single-bit error conceivably could corrupt the entire compressed archive". Tar/gzip archives are great for distributing source packages, where you can simply get a new copy if yours goes bad, but they're not as good for backups.
Bit errors in a zip archive typically will make only the one file unrecoverable. And note that bit errors may develop over time -- just because your archive passes the md5sum check today, it might not in a year. CDs have an expected shelf life of 100 years, but remember, that hard drive that failed had a MTBF of 100,000 hours.
True
Yea I have yet to recover a bad file yet!
mbr equivalent for lvm?
What would be the equivalent to the 'dd' statement for storing the layout of an lvm setup? Is it stored i a regular file, or is there a chunk of disk that needs copying?
(I don't have access to a linux box at the moment to investigate the answer for myself.)
Rsnapshot
How could you have not mentioned rsnapshot? ;-)
www.rsnapshot.org
Good tips on rolling your own
Thanks for a fine article on how to roll your own solution.
I have been using Hugo Rabson's MONDO from www.mondorescue.org to create backup disks for years. I think Hugo's code started with what you are doing here but has expanded. It still retains the kind of flexability and ease of use for personal backups. It has the capability to create CD-R, CD-RW, DVD-RW, ISO images, backups to NFS volumes and even to tape.
A CD-RW is far too small.
Im sure it's nice, and all, but what about people with 160GB drives?
My collection of digital photos I've taken takes about 10GB alone.
I just have to trust in XFS and Western Digital :(
Well, if you value your
Well, if you value your data, you should be buying a backup solution as part of your computer investment. So, if you can spend extra on a 300G hard drive, you should probably instead buy 2 160G hard drives, for instance. The price vs. size vs. quantity thing isn't quite that simple, but you get the idea. Backup solutions are not an "extra" thing; they're part of your basic costs. If you can't afford a backup system, then you can't afford the computer at all.
But, at the very least, partition the drive, and backups to the other partition with rsync or something like dirvish, as mentioned elsewhere.
re... too small
I have the same problem. I have about 2 GB of saved docs and email, 3 GB of digital pics and about 10 GB of music that I don't want to get lost. I have the cd's for most of the music (yes I do own it) but to get it back onto the hard disk I would have to rip for a week.
My solution:
1) Grab an old pc out of the garage (PPro 200MZ in this case)
2) Install 3 of the biggest disk drives I had laying around.
2) Install Linux (RedHat 7.3 with all the Fedora Lagacy patches)
3) Plug it into the network. ( We now have NAS)
4) Set up rsync to backup nightly. (rsync only sends files that have changed)
5) Set up Samba to allow the wife to save her "My Documents"
Scripts:
$ cat sunday_sync
#!/bin/bash
sun_sync_doc_with_ast
sun_sync_pic_with_ast
sync_dudley_with_ast
sync_mp3_with_ast
$ cat sun_sync_doc_with_ast
#!/bin/bash
cd /home/me/
rsync -avz ./Documents me@192.168.1.50:./Doc1
$ cat sync_mp3_with_ast
#!/bin/bash
cd /home/me/Music
rsync -avz ./mp3 me@192.168.1.50:./Music
(you get the idea... I rsync nightly to this machine. I also have more then one directory on the target machine for doc and pic ... ie. ./Doc1 ./Doc2 ./Doc3 .... and rotate through the week. )
Notes:
You will notice that I have several copies of ./Doc on the backup server but only one copy of ./Music . Music doesn't change that often and if I did lose it it would be more of a hassle then a disaster. (You get extra points for making sure that ./Doc1 is on a different hard disk then ./Doc2 is on a different hard disk then ./Doc3)
In order to make the rsync happen with out my having to imput passwords I created an ssh key with no password. This is a little bit of a security hole but since this is sitting in my home office and you would have to hack my main pc to access the key... oh well. Even if you did this key is only used for the backup machine.
I choose RedHat 7.3 for the following reasons.
1) I had the disks sitting in my drawer.
2) With the Fedora Lagacy project putting out security updates it is as secure as any other version of RedHat.
3) Version 7.3 runs great on this old Pentium Pro 200. 7.3 was the high point for RedHat. It was fast, complete, stable, and ran well. Every version since has gotten more and more bloated and slower and slower on anything except "state of the art" hardware. In my opinion they are close to if not already living in Microsoft Bloatsville. RedHat needs to force their developers use 500MZ Pentium II machines for their desktops.
if you want rsync web
if you want rsync web interface or gui tools check this very useful
rsync web interface or gui tools
Have a look at dirvish.you
Have a look at dirvish.
you can use it on your local computer and via the network. it also uses rsync for transferring the data, but you can keep multiple backups of your data wihout wasting space for common files. this is achieved by using hardlinks for common files. it also provides indexing and search functionality. I use it for several months on our servers and had no problems...
Yes. Something like Dirvish
Yes. Something like Dirvish is ideal, I think. Optical media is far too unreliable for backups, imho. At best, they're useful for shipping data or transferring to a non-networked machine. I was using faubackup for a while, to transfer from one machine's HD to another's, but dirvish is similar, only faster; just got finished setting it up myself. It's a little more complex than faubackup (which would still be useful for very simple backups or backups to the same machine), but not too difficult to get your head around, and quite flexible and powerful too.
A CD-RW is far too small.
For large amounts of data, an external USB2 or firewire drive is the only practical way to go.
It should be possible to modify this script to use an external USB disk instead. For rotation and "security" use multiple directories or partitions.
I think you switched the bs
I think you switched the bs and count #'s to restore the mbr.
D#ont use a CD-RW
Correct, the only way is to use an external USB2, don't use a CD-RW, it's not suitable for data in large amounts.
External USb2 and firewir drive
Hi Bob, you are correct in your statement, but a firewire drive is also
possible.
Best regards Hilde from Germany
I read some articles and
I read some articles and learned a lot from your journal. Thank you and success for the future.
Best wishes
Dobby
Interesting articles
By reviewing your articles I can found out the same as Dobby(see his comment on September 9). I like your journal, especially the content but also the design.
Success in the future
Carsten
Thanks
A helpful script for external USB!
By modifying the script use
By modifying the script use an external USB disk instead.
best regards
Heinz
my comment before
sorry, my homepage behind my name heinz in not running. therefore
i try it again.
best regards
heinz
Easy backup
Easy backup and restore is a neverending story. I like the comments, helpful for my work.