Mirrordir Can Make You a Hometown Hero!
January 20th, 2003 by Jeffrey McDonald in
What is the least attractive task of a system administrator's job? For most of us, it's performing backups of our data. Something about backups leaves us yawning and wishing for something far more exciting to do. Even with the advent of ATLs and backup software automation, we still need to watch log files and test the backup media occasionally by restoring files to assure the data is intact. Yep, backups are a necessary pain--but what if we could supplement our work with an extra layer of protection that, when disaster does strike, allows us to recover date much faster than if we went back to the tape?
If you haven't heard of the mirrordir utility, do yourself a favor and check it out. For one thing, it's extremely simple to use, but it's also powerful. It is a tool no system administrator should be without.
Let's take a look at what mirrordir can do for us. First, if you haven't already done so, download mirrordir from mirrordir.sourceforge.net and choose from either the RPM or tar version. I prefer to use RPMs when they are available, so let's go ahead and install it as the root user, from the directory you downloaded it to, with rpm -ihv mirrordir*.rpm. Provided no dependency issues halted the installation (as of yet, I have found none with either Red Hat 7.2 or 8.0 installations), we can proceed.
What you choose to do with mirrordir depends on you. It's purpose in life is to make an exact mirror of a directory or directories and copy all data to a safe mirrored location. This can be a location on the same disk, on the same server on a separate disk, on an FTP site or on any other network storage site. Obviously, mirroring data to the same disk is not the best idea should the disk itself fail, but it can be done if you desire.
For our example, we will set up mirrordir to copy the exact contents from one external Compaq SCSI RAID cabinet to another identical external cabinet (like to like). Also, we will write a simple script to automate the task of keeping the data up-to-date and add this script to our weekly cron jobs (or daily jobs, if you prefer). Each external RAID cabinet, /dev/sda and /dev/sdb, houses six SCSI drives set up as a RAID-5 array. Once formatted as an ext3 filesystem, we have approximately 90GB of storage per cabinet.
We begin by creating a mountpoint for the external cabinets. As the root user, create a directory called ext_raid and another called mirror (or whatever you wish to call them). Next set the permissions on these mountpoints as follows, chmod -R 755. Once the mountpoints are in place and the permissions are set up as you like, do a dry run with mirrordir. First, assure that the two external SCSI arrays are mounted by issuing the mount -a command (which assumes that the external arrays have been added to the /etc/fstab file for mounting). Once both devices are mounted, run a mirrordir test:
mirrordir -v --dry-run /dev/sda /mirror
Pretty simple command line, huh? Don't be fooled though, as this is a very useful program. In our first example we used the dry-run switch, which simply lists the output of what would have been done. Before actually running mirrordir on your data, familiarize yourself with the man page listing all the available options. And, of course, have a current (yep, here it comes) backup available in case of an unforeseen booboo.
A word of caution here: be very careful with the order of the parameters you give to mirrordir. Should you reverse the order, you could replace data you are trying to make a copy of with data that exists on the target device--or you could delete everything on the source. This is one reason why you should do a dry-run first and examine all of the output mirrordir produces. That way, you are sure you are sending the correct data to the correct location. How are your backups?
Now let's run mirrordir for real. Copy the source data from one external RAID cabinet, called ext_raid, to a second cabinet, called mirror. This data consists of tons of technical documents, MP3s, applications, utilities and video from my SnapStream PVS server. Mirrordir will then make an exact copy of all my directories and data on the target RAID array (/mirror), maintaining the original security settings for the files and directories.
From this point forward, mirrordir needs only to append any changes to the data rather than do a full copy each time it is run. This feature is especially nice if you are mirroring over an FTP link across the network. Again, be sure to check out all of the cool options available in mirrordir. To name a few, you can use gzip to compress the data and create a tarball of the data on the target destination.
Now that we know mirrordir is working as promised, we can write a simple script to mount the target drive array, mirror our data to it, then unmount the array. This way the mirror disks are off-line and cannot be accidentally erased or modified. Open up your favorite editor and add something similar to this:
#!/bin/sh # This is mirror.sh - copy data from ext_raid cabinet to the mirror cabinet. # # this is mirrordir.sh: mirror of /ext_raid to another external raid cabinets # we assume that the source /ext_raid is already mounted, so we just mount the mirror /bin/mount /dev/sdb /mirror mirrordir /ext_raid /mirror /bin/umount /mirror
Next save the script and run chmod to make it executable by issuing the following:
chmod u+x mirror.sh
Now you can add the script to an existing cron job by dropping the script into one of the cron directories in /etc/cron.d (daily, weekly and so on).
So what about recovering from pooched data or a dead storage device? Once your target device is back up and running, simply reverse the previous mirrordir command:
mount /dev/sdb /mirror mirrordir /mirror /ext_mirror
Watch and smile as an exact copy of your data is fully restored to your (now repaired) original RAID cabinet or to the appropriate target destination. Of course, you also can point users to the /mirror cabinet, until the failed device is ready. Do I smell another script here?.
That's all there is to it, really. Mirrordir has additional functionality, though, so check out the man page or the mirrordir.sourceforge.net home page for additional samples. Mirrordir is yet another utility that further demonstrates the power of Linux and the thousands of developers that contribute.
Jeffrey McDonald is a UNIX/Linux systems engineer working at a large Fortune 500 technology company in central California.
Special Magazine Offer -- 2 Free Trial Issues!
Receive 2 free trial issues of Linux Journal as well as instant online access to current and past issues. There's NO RISK and NO OBLIGATION to buy. CLICK HERE for offer
Linux Journal: delivering readers the advice and inspiration they need to get the most out of their Linux systems since 1994.
Sorry, offer available in the US only. International orders, click here.
Subscribe now!
The Latest
Featured Videos
Linux Journal Live - Oct 9, 2008
October 9th, 2008 by Shawn Powers
The October 9, 2008 edition of Linux Journal Live! Associate Editor, Shawn Powers, and Kyle Rankin, "Hack and /" columnist and author of Knoppix Hacks, Linux Multimedia Hacks, Knoppix Pocket Reference and others, discuss Linux distributions.
Linux Journal Live - Oct 2, 2008
October 3rd, 2008 by Shawn Powers
The October 2, 2008 edition of Linux Journal Live! Associate Editor, Shawn Powers, and Steven Evatt, Online Development manager for The Houston Chronicle discuss surviving disaster with Linux.
Recently Popular
From the Magazine
November 2008, #175
There aren't many numbers that put the US national debt to shame, but here's one: 1,100,000,000,000,000. What's that? That's how many floating-point operations per second the Roadrunner supercomputer at Las Alamos can perform. That's about 100 FLOPS per dollar of US debt (unfortunately, the debt is winning the second derivative race). Read the article about Roadrunner in this month's High Performance Computing issue of LJ.
Along with that, find out how to program the Cell processor and how to use CUDA with your NVIDIA GPU. Also in this issue: Mr HandS (aka Kyle Rankin) gives us a few tips on using Compiz, Chef Marcel shows you how to get blogging off your plate quicker, Mick Bauer talks about Samba security, Dan Sawyer interviews Cory Doctrow and Doc talks about how information technology can affect democracy and fix the national debt (just kidding about that last part). That and more for your reading pleasure in this month's Linux Journal.
Delicious
Digg
Reddit
Newsvine
Technorati








Re: Mirrordir Can Make You a Hometown Hero!
On January 20th, 2003 Anonymous says:
Have you looked at rsync?
http://rsync.samba.org
The hardlink trick makes hourly/daily backups trivial, fast and space efficient. http://www.mikerubel.org/computers/rsync_snapshots/
Post new comment