Hack and / - Lightning Hacks
Recently, I was working on a remastered Knoppix DVD that I had sent out to a few people. After I had sent out the full remastered DVD, I found out that I needed to change a few small files on the DVD. Even though my home DSL speeds are pretty fast, the upload is still slow enough that it took overnight to transfer the 3GB+ DVD image. I didn't want to go through that again, especially as I had made only minor changes to the DVD.
I knew that binary diff tools existed, but I discovered that not all of them are equal. Some binary diff tools require enough RAM to store multiple copies of the file, which certainly wouldn't work with a 3GB image. Lucky for me, I found rdiff, a tool that works well with large files and doesn't require a lot of RAM. What's better is that rdiff works with any binary—you can use it for any large binary files from DVD images to virtual disks to multimedia files.
rdiff works via a three-stage process. In this example, I have two files, old.iso and new.iso, that have minor differences from each other. For the first stage, you create a signature file that rdiff uses to represent your original file:
$ rdiff signature old.iso old.signature
Now that you have a signature file, use it with rdiff to create a delta file that represents the differences between the old and new files:
$ rdiff delta old.signature new.iso new.delta
This new.delta file is now all that anyone needs to convert old.iso to new.iso. For me, this file ended up being around 150Kb, because I had made only a few changes. The delta file was much simpler to send around than the full image. If you want to test that the delta file will work, first create an md5sum of new.iso:
$ md5sum new.iso
Then, use rdiff to patch the old file with the delta to create the new file. This is the same command that everyone else with the original file will use:
$ rdiff patch odl.iso new.delta newtest.iso
Now that you have newtest.iso, create an md5sum of that file and compare it with the one you made for new.iso:
$ md5sum newtest.iso
As I said before, this method works not only with ISOs, but also with any binary file large or small. It's worth noting that rdiff works with the same binary diff method rsync uses. rdiff just lets you use the algorithm step by step on the command line.
Kyle Rankin is a Senior Systems Administrator in the San Francisco Bay Area and the author of a number of books, including Knoppix Hacks and Ubuntu Hacks for O'Reilly Media. He is currently the president of the North Bay Linux Users' Group.
Kyle Rankin is a director of engineering operations in the San Francisco Bay Area, the author of a number of books including DevOps Troubleshooting and The Official Ubuntu Server Book, and is a columnist for Linux Journal.
Getting Started with DevOps - Including New Data on IT Performance from Puppet Labs 2015 State of DevOps Report
August 27, 2015
12:00 PM CDT
DevOps represents a profound change from the way most IT departments have traditionally worked: from siloed teams and high-anxiety releases to everyone collaborating on uneventful and more frequent releases of higher-quality code. It doesn't matter how large or small an organization is, or even whether it's historically slow moving or risk averse — there are ways to adopt DevOps sanely, and get measurable results in just weeks.
Free to Linux Journal readers.Register Now!
- Django Models and Migrations
- Hacking a Safe with Bash
- Secure Server Deployments in Hostile Territory, Part II
- The Controversy Behind Canonical's Intellectual Property Policy
- Huge Package Overhaul for Debian and Ubuntu
- Home Automation with Raspberry Pi
- Shashlik - a Tasty New Android Simulator
- Embed Linux in Monitoring and Control Systems
- KDE Reveals Plasma Mobile
- diff -u: What's New in Kernel Development