File Synchronization with Unison
Unison is a file-synchronization tool that runs on Linux, UNIX and Microsoft Windows. Those of you who've used IBM Lotus Notes or Intellisync Mobile Suite probably have an idea of what synchronization is good for, as compared to one-way mirroring options such as rsync. You might have mirrored a company document directory to your laptop, for example, and then modified a document or two. Other people might have modified other documents in the same directory by the time you get back. With rsync, you'd need to reconcile the differences between the two directories manually or risk overwriting someone's changes. Unison can sort out what has changed where, propagate the changed files and even merge different changes to the same file if you tell it how.
Think of Unison as two-way rsync with a bit of revision control mixed in. The most common use is keeping your local and remote home directory, or some data directory you often use in different contexts, in sync. It uses the rsync algorithm to keep network traffic down and should be tunneled through SSH over untrusted networks. No extra work is needed—simply specify ssh:// when adding a directory location. Quite a bit of extra disk space often is needed for Unison, though, because the synchronizer needs to keep track of what the files looked like on the last run.
Unison's home page is maintained at the University of Pennsylvania; the project leader, Benjamin C. Pierce, is a professor in the Department of Computer and Information Science. See the on-line Resources for the URL.
Unison isn't as widely deployed as rsync, so you might not be able to find a precompiled package for your distribution. But the binaries downloadable from the Unison home page should work for most people.
If you'd like to compile from source, you can. A few extra hoops must be jumped through, however, because Unison is programmed in OCaml, not the most common language. See Resources if there is no handy package for your distribution.
Compiling and installing Unison is simple; type make UISTYLE=xxx. The GTK user interface needs additional OCaml bindings for GTK, so I use the text interface in this article. Typing make UISTYLE=text or make UISTYLE=gtk should give you a Unison executable. Simply copy the executable to somewhere in the path on both machines you want to synchronize.
In this article, I'm using the current stable version of Unison, 2.9.1, unless otherwise noted. You need to use the latest betas if you're going to synchronize files larger than 2GB.
The developer versions tend to work well. They are what the developers run themselves on their own precious data. Sign up for the unison-hackers mailing list if you feel a bit adventurous. Jerome Vouillon, Benjamin C. Pierce and Trevor Jim tend to hang out there discussing improvements. Commit logs also float by, so you can track what is going on.
Unison keeps its config and working files in a .unison directory in your home directory or wherever you want to put it. Set the UNISON environment variable to specify an alternate location.
The default configuration is stored in .unison/default.prf. Listing 1 shows a plain config file suitable for testing. Synchronizing two directories is now as simple as:
$ unison /nfsmount/dir1 /home/me/dir1
Listing 1. .unison/default.prf
# Unison preferences file merge = diff3 -m CURRENT1 OLD CURRENT2 > NEW backup = Name * maxbackups = 10 log = true logfile = /home/knan/.unison/unison.log rshargs = -C
Unison then asks the user about any differences between the directories and offers reasonable defaults. It does take a bit of time to get used to Unison's way of thinking, however. And, Unison is no substitute for backups. Unison happily propagates back the deletion of all the files in one replica, for example, which can be a rude awakening for programmers used to CVS. For example:
rm dir1/* ; unison ssh://server/dir1 dir1
Deleting a file is an action that is replicated on the other side upon synchronization. So, this example command removes all files in dir1 on both sides.
Once you feel comfortable, consider adding auto = true to the Unison profile. This skips questions about any non-conflicting changes but gives you a chance to back out at the end.
The Unison manual is recommended reading. It is clear and well written and explains what happens at most corner cases.
Free Webinar: Hadoop
How to Build an Optimal Hadoop Cluster to Store and Maintain Unlimited Amounts of Data Using Microservers
Realizing the promise of Apache® Hadoop® requires the effective deployment of compute, memory, storage and networking to achieve optimal results. With its flexibility and multitude of options, it is easy to over or under provision the server infrastructure, resulting in poor performance and high TCO. Join us for an in depth, technical discussion with industry experts from leading Hadoop and server companies who will provide insights into the key considerations for designing and deploying an optimal Hadoop cluster.
Some of key questions to be discussed are:
- What is the “typical” Hadoop cluster and what should be installed on the different machine types?
- Why should you consider the typical workload patterns when making your hardware decisions?
- Are all microservers created equal for Hadoop deployments?
- How do I plan for expansion if I require more compute, memory, storage or networking?
|Speed Up Your Web Site with Varnish||Jun 19, 2013|
|Non-Linux FOSS: libnotify, OS X Style||Jun 18, 2013|
|Containers—Not Virtual Machines—Are the Future Cloud||Jun 17, 2013|
|Lock-Free Multi-Producer Multi-Consumer Queue on Ring Buffer||Jun 12, 2013|
|Weechat, Irssi's Little Brother||Jun 11, 2013|
|One Tail Just Isn't Enough||Jun 07, 2013|
- Speed Up Your Web Site with Varnish
- Containers—Not Virtual Machines—Are the Future Cloud
- Linux Systems Administrator
- Lock-Free Multi-Producer Multi-Consumer Queue on Ring Buffer
- Non-Linux FOSS: libnotify, OS X Style
- Senior Perl Developer
- Technical Support Rep
- UX Designer
- RSS Feeds
- Reply to comment | Linux Journal
37 min 15 sec ago
- Reply to comment | Linux Journal
4 hours 36 min ago
- Yeah, user namespaces are
5 hours 53 min ago
- Cari Uang
9 hours 24 min ago
- user namespaces
12 hours 18 min ago
12 hours 43 min ago
- One advantage with VMs
15 hours 12 min ago
- about info
15 hours 45 min ago
15 hours 46 min ago
15 hours 47 min ago