File Synchronization with Unison
Unison is a file-synchronization tool that runs on Linux, UNIX and Microsoft Windows. Those of you who've used IBM Lotus Notes or Intellisync Mobile Suite probably have an idea of what synchronization is good for, as compared to one-way mirroring options such as rsync. You might have mirrored a company document directory to your laptop, for example, and then modified a document or two. Other people might have modified other documents in the same directory by the time you get back. With rsync, you'd need to reconcile the differences between the two directories manually or risk overwriting someone's changes. Unison can sort out what has changed where, propagate the changed files and even merge different changes to the same file if you tell it how.
Think of Unison as two-way rsync with a bit of revision control mixed in. The most common use is keeping your local and remote home directory, or some data directory you often use in different contexts, in sync. It uses the rsync algorithm to keep network traffic down and should be tunneled through SSH over untrusted networks. No extra work is needed—simply specify ssh:// when adding a directory location. Quite a bit of extra disk space often is needed for Unison, though, because the synchronizer needs to keep track of what the files looked like on the last run.
Unison's home page is maintained at the University of Pennsylvania; the project leader, Benjamin C. Pierce, is a professor in the Department of Computer and Information Science. See the on-line Resources for the URL.
Unison isn't as widely deployed as rsync, so you might not be able to find a precompiled package for your distribution. But the binaries downloadable from the Unison home page should work for most people.
If you'd like to compile from source, you can. A few extra hoops must be jumped through, however, because Unison is programmed in OCaml, not the most common language. See Resources if there is no handy package for your distribution.
Compiling and installing Unison is simple; type make UISTYLE=xxx. The GTK user interface needs additional OCaml bindings for GTK, so I use the text interface in this article. Typing make UISTYLE=text or make UISTYLE=gtk should give you a Unison executable. Simply copy the executable to somewhere in the path on both machines you want to synchronize.
In this article, I'm using the current stable version of Unison, 2.9.1, unless otherwise noted. You need to use the latest betas if you're going to synchronize files larger than 2GB.
The developer versions tend to work well. They are what the developers run themselves on their own precious data. Sign up for the unison-hackers mailing list if you feel a bit adventurous. Jerome Vouillon, Benjamin C. Pierce and Trevor Jim tend to hang out there discussing improvements. Commit logs also float by, so you can track what is going on.
Unison keeps its config and working files in a .unison directory in your home directory or wherever you want to put it. Set the UNISON environment variable to specify an alternate location.
The default configuration is stored in .unison/default.prf. Listing 1 shows a plain config file suitable for testing. Synchronizing two directories is now as simple as:
$ unison /nfsmount/dir1 /home/me/dir1
Listing 1. .unison/default.prf
# Unison preferences file merge = diff3 -m CURRENT1 OLD CURRENT2 > NEW backup = Name * maxbackups = 10 log = true logfile = /home/knan/.unison/unison.log rshargs = -C
Unison then asks the user about any differences between the directories and offers reasonable defaults. It does take a bit of time to get used to Unison's way of thinking, however. And, Unison is no substitute for backups. Unison happily propagates back the deletion of all the files in one replica, for example, which can be a rude awakening for programmers used to CVS. For example:
rm dir1/* ; unison ssh://server/dir1 dir1
Deleting a file is an action that is replicated on the other side upon synchronization. So, this example command removes all files in dir1 on both sides.
Once you feel comfortable, consider adding auto = true to the Unison profile. This skips questions about any non-conflicting changes but gives you a chance to back out at the end.
The Unison manual is recommended reading. It is clear and well written and explains what happens at most corner cases.
Free DevOps eBooks, Videos, and more!
Regardless of where you are in your DevOps process, Linux Journal can help!
We offer here the DEFINITIVE DevOps for Dummies, a mobile Application Development Primer, and advice & help from the expert sources like:
- Linux Journal
- New Products
- Users, Permissions and Multitenant Sites
- Flexible Access Control with Squid Proxy
- Security in Three Ds: Detect, Decide and Deny
- High-Availability Storage with HA-LVM
- DevOps: Everything You Need to Know
- Tighten Up SSH
- Non-Linux FOSS: MenuMeters
- Solving ODEs on Linux
- diff -u: What's New in Kernel Development