Mirrordir Can Make You a Hometown Hero!
What is the least attractive task of a system administrator's job? For most of us, it's performing backups of our data. Something about backups leaves us yawning and wishing for something far more exciting to do. Even with the advent of ATLs and backup software automation, we still need to watch log files and test the backup media occasionally by restoring files to assure the data is intact. Yep, backups are a necessary pain--but what if we could supplement our work with an extra layer of protection that, when disaster does strike, allows us to recover date much faster than if we went back to the tape?
If you haven't heard of the mirrordir utility, do yourself a favor and check it out. For one thing, it's extremely simple to use, but it's also powerful. It is a tool no system administrator should be without.
Let's take a look at what mirrordir can do for us. First, if you haven't already done so, download mirrordir from mirrordir.sourceforge.net and choose from either the RPM or tar version. I prefer to use RPMs when they are available, so let's go ahead and install it as the root user, from the directory you downloaded it to, with rpm -ihv mirrordir*.rpm. Provided no dependency issues halted the installation (as of yet, I have found none with either Red Hat 7.2 or 8.0 installations), we can proceed.
What you choose to do with mirrordir depends on you. It's purpose in life is to make an exact mirror of a directory or directories and copy all data to a safe mirrored location. This can be a location on the same disk, on the same server on a separate disk, on an FTP site or on any other network storage site. Obviously, mirroring data to the same disk is not the best idea should the disk itself fail, but it can be done if you desire.
For our example, we will set up mirrordir to copy the exact contents from one external Compaq SCSI RAID cabinet to another identical external cabinet (like to like). Also, we will write a simple script to automate the task of keeping the data up-to-date and add this script to our weekly cron jobs (or daily jobs, if you prefer). Each external RAID cabinet, /dev/sda and /dev/sdb, houses six SCSI drives set up as a RAID-5 array. Once formatted as an ext3 filesystem, we have approximately 90GB of storage per cabinet.
We begin by creating a mountpoint for the external cabinets. As the root user, create a directory called ext_raid and another called mirror (or whatever you wish to call them). Next set the permissions on these mountpoints as follows, chmod -R 755. Once the mountpoints are in place and the permissions are set up as you like, do a dry run with mirrordir. First, assure that the two external SCSI arrays are mounted by issuing the mount -a command (which assumes that the external arrays have been added to the /etc/fstab file for mounting). Once both devices are mounted, run a mirrordir test:
mirrordir -v --dry-run /dev/sda /mirror
Pretty simple command line, huh? Don't be fooled though, as this is a very useful program. In our first example we used the dry-run switch, which simply lists the output of what would have been done. Before actually running mirrordir on your data, familiarize yourself with the man page listing all the available options. And, of course, have a current (yep, here it comes) backup available in case of an unforeseen booboo.
A word of caution here: be very careful with the order of the parameters you give to mirrordir. Should you reverse the order, you could replace data you are trying to make a copy of with data that exists on the target device--or you could delete everything on the source. This is one reason why you should do a dry-run first and examine all of the output mirrordir produces. That way, you are sure you are sending the correct data to the correct location. How are your backups?
Now let's run mirrordir for real. Copy the source data from one external RAID cabinet, called ext_raid, to a second cabinet, called mirror. This data consists of tons of technical documents, MP3s, applications, utilities and video from my SnapStream PVS server. Mirrordir will then make an exact copy of all my directories and data on the target RAID array (/mirror), maintaining the original security settings for the files and directories.
From this point forward, mirrordir needs only to append any changes to the data rather than do a full copy each time it is run. This feature is especially nice if you are mirroring over an FTP link across the network. Again, be sure to check out all of the cool options available in mirrordir. To name a few, you can use gzip to compress the data and create a tarball of the data on the target destination.
Now that we know mirrordir is working as promised, we can write a simple script to mount the target drive array, mirror our data to it, then unmount the array. This way the mirror disks are off-line and cannot be accidentally erased or modified. Open up your favorite editor and add something similar to this:
#!/bin/sh # This is mirror.sh - copy data from ext_raid cabinet to the mirror cabinet. # # this is mirrordir.sh: mirror of /ext_raid to another external raid cabinets # we assume that the source /ext_raid is already mounted, so we just mount the mirror /bin/mount /dev/sdb /mirror mirrordir /ext_raid /mirror /bin/umount /mirror
Next save the script and run chmod to make it executable by issuing the following:
chmod u+x mirror.sh
Now you can add the script to an existing cron job by dropping the script into one of the cron directories in /etc/cron.d (daily, weekly and so on).
So what about recovering from pooched data or a dead storage device? Once your target device is back up and running, simply reverse the previous mirrordir command:
mount /dev/sdb /mirror mirrordir /mirror /ext_mirror
Watch and smile as an exact copy of your data is fully restored to your (now repaired) original RAID cabinet or to the appropriate target destination. Of course, you also can point users to the /mirror cabinet, until the failed device is ready. Do I smell another script here?.
Practical Task Scheduling Deployment
One of the best things about the UNIX environment (aside from being stable and efficient) is the vast array of software tools available to help you do your job. Traditionally, a UNIX tool does only one thing, but does that one thing very well. For example, grep is very easy to use and can search vast amounts of data quickly. The find tool can find a particular file or files based on all kinds of criteria. It's pretty easy to string these tools together to build even more powerful tools, such as a tool that finds all of the .log files in the /home directory and searches each one for a particular entry. This erector-set mentality allows UNIX system administrators to seem to always have the right tool for the job.
Cron traditionally has been considered another such a tool for job scheduling, but is it enough? This webinar considers that very question. The first part builds on a previous Geek Guide, Beyond Cron, and briefly describes how to know when it might be time to consider upgrading your job scheduling infrastructure. The second part presents an actual planning and implementation framework.
Join Linux Journal's Mike Diehl and Pat Cameron of Help Systems.
Free to Linux Journal readers.View Now!
|The Firebird Project's Firebird Relational Database||Jul 29, 2016|
|Stunnel Security for Oracle||Jul 28, 2016|
|SUSE LLC's SUSE Manager||Jul 21, 2016|
|My +1 Sword of Productivity||Jul 20, 2016|
|Non-Linux FOSS: Caffeine!||Jul 19, 2016|
|Murat Yener and Onur Dundar's Expert Android Studio (Wrox)||Jul 18, 2016|
- Stunnel Security for Oracle
- The Firebird Project's Firebird Relational Database
- Murat Yener and Onur Dundar's Expert Android Studio (Wrox)
- SUSE LLC's SUSE Manager
- Managing Linux Using Puppet
- My +1 Sword of Productivity
- Non-Linux FOSS: Caffeine!
- Google's SwiftShader Released
- SuperTuxKart 0.9.2 Released
- Doing for User Space What We Did for Kernel Space
With all the industry talk about the benefits of Linux on Power and all the performance advantages offered by its open architecture, you may be considering a move in that direction. If you are thinking about analytics, big data and cloud computing, you would be right to evaluate Power. The idea of using commodity x86 hardware and replacing it every three years is an outdated cost model. It doesn’t consider the total cost of ownership, and it doesn’t consider the advantage of real processing power, high-availability and multithreading like a demon.
This ebook takes a look at some of the practical applications of the Linux on Power platform and ways you might bring all the performance power of this open architecture to bear for your organization. There are no smoke and mirrors here—just hard, cold, empirical evidence provided by independent sources. I also consider some innovative ways Linux on Power will be used in the future.Get the Guide