Tarsnap: On-line Backups for the Truly Paranoid

Storing backups in the cloud requires a level of trust that not everyone is willing to give. While the convenience and low cost of automated, off-site backups is very compelling, the reality of putting personal data in the hands of complete strangers will never sit quite right with some people.

Enter Tarsnap—"on-line backups for the truly paranoid". Tarsnap is the brainchild of Dr Colin Percival, a former FreeBSD Security Officer. In 2006, he began research and development on a new solution for "encrypted, snapshotted remote backups", culminating in the release of Tarsnap in 2008.

Unlike other on-line backup solutions, Tarsnap uses an open, documented cryptographic design that securely encrypts your files. Rather than trusting a vendor's cryptographic claims, you have full access to the source code, which uses open-source libraries and industry-vetted protocols, such as RSA, AES and SHA.

Tarsnap provides a command-line client that operates very much like the traditional UNIX tar command. Familiar syntax, such as tarsnap cf and tarsnap xvf works as users would expect, except that instead of manipulating local tarballs, the client is working with cloud-based archives. These archives are stored on Amazon S3 with EC2 servers to handle client connections. To minimize bandwidth costs, Tarsnap uses smart rsync-like block-oriented snapshot operations that upload only data set changes and minimize transmission costs.

Although you could roll your own backup system by encrypting your data, running an rsync server off-site and so on, Tarsnap's scriptability, professional cryptographic design and utility pricing make it an attractive service for clueful users.

Open-Source Strong Crypto

Tarsnap compresses, encrypts and cryptographically signs every byte you send to it. No knowledge of cryptographic protocols is required, but if you are an interested enthusiast or feel more comfortable if you can look under the hood, you'll find the source code and full design thoroughly documented on http://tarsnap.com.

During installation, the user creates a keyfile, which optionally can be passphrase-protected. The file contains two 2048-bit RSA keys for signing and encryption. AES-256 session keys are created at runtime and encrypted with the RSA keys. The HMAC-SHA-256 hash function is used to verify data blocks and prevent tampering, and the same hash function is used elsewhere to verify communication between the client and server.

If those cryptographic terms make your eyes glaze over, the simple description is that Tarsnap encrypts and verifies your data in motion and at rest using industry-standard protocols. From a user perspective, the only thing that needs to be safeguarded is the keyfile. As you might expect, if it's lost, the keys to decrypt data stored with Tarsnap are unavailable and your data is no longer usable. After generation, this keyfile ideally should be stored securely off-site.

Efficient Design

Tarsnap backs up variable-length, deduplicated blocks of data. This means that if you change one file in a directory, only that file is backed up. Indeed, only the changes to that file are backed up if the file previously existed. If you move files around, Tarsnap is smart enough to recognize and skip previously backed-up blocks. Anytime you wish, you can take a backup of a directory, which creates a new archive. Behind the scenes, Tarsnap will retain whatever files are necessary to make that archive whole.

Suppose you have a directory with 100MB of files and a 10% daily change rate. If you create a Tarsnap archive every day of that directory, you will see the following consumption pattern:

• Monday (initial): 100MB archive created (100MB uploaded, 100MB stored).

• Tuesday: 10MB archive created (10MB new data uploaded, 110MB total data stored).

• Wednesday: 10MB archive created (10MB new data uploaded, 120MB total data stored).

If on Thursday you delete the Monday archive, the Tuesday archive will be "made whole" by retaining whatever files from Monday are needed. This gives you tremendous flexibility to pick restore points. Of course, you can keep multiple archives with however many restore points you wish.