Hack and / - Lightning Hacks
Recently, I was working on a remastered Knoppix DVD that I had sent out to a few people. After I had sent out the full remastered DVD, I found out that I needed to change a few small files on the DVD. Even though my home DSL speeds are pretty fast, the upload is still slow enough that it took overnight to transfer the 3GB+ DVD image. I didn't want to go through that again, especially as I had made only minor changes to the DVD.
I knew that binary diff tools existed, but I discovered that not all of them are equal. Some binary diff tools require enough RAM to store multiple copies of the file, which certainly wouldn't work with a 3GB image. Lucky for me, I found rdiff, a tool that works well with large files and doesn't require a lot of RAM. What's better is that rdiff works with any binary—you can use it for any large binary files from DVD images to virtual disks to multimedia files.
rdiff works via a three-stage process. In this example, I have two files, old.iso and new.iso, that have minor differences from each other. For the first stage, you create a signature file that rdiff uses to represent your original file:
$ rdiff signature old.iso old.signature
Now that you have a signature file, use it with rdiff to create a delta file that represents the differences between the old and new files:
$ rdiff delta old.signature new.iso new.delta
This new.delta file is now all that anyone needs to convert old.iso to new.iso. For me, this file ended up being around 150Kb, because I had made only a few changes. The delta file was much simpler to send around than the full image. If you want to test that the delta file will work, first create an md5sum of new.iso:
$ md5sum new.iso
Then, use rdiff to patch the old file with the delta to create the new file. This is the same command that everyone else with the original file will use:
$ rdiff patch odl.iso new.delta newtest.iso
Now that you have newtest.iso, create an md5sum of that file and compare it with the one you made for new.iso:
$ md5sum newtest.iso
As I said before, this method works not only with ISOs, but also with any binary file large or small. It's worth noting that rdiff works with the same binary diff method rsync uses. rdiff just lets you use the algorithm step by step on the command line.
Kyle Rankin is a Senior Systems Administrator in the San Francisco Bay Area and the author of a number of books, including Knoppix Hacks and Ubuntu Hacks for O'Reilly Media. He is currently the president of the North Bay Linux Users' Group.
Kyle Rankin is a VP of engineering operations at Final, Inc., the author of a number of books including DevOps Troubleshooting and The Official Ubuntu Server Book, and is a columnist for Linux Journal. Follow him @kylerankin.
Fast/Flexible Linux OS Recovery
On Demand Now
In this live one-hour webinar, learn how to enhance your existing backup strategies for complete disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible full-system recovery solution for UNIX and Linux systems.
Join Linux Journal's Shawn Powers and David Huffman, President/CEO, Storix, Inc.
Free to Linux Journal readers.Register Now!
- Download "Linux Management with Red Hat Satellite: Measuring Business Impact and ROI"
- Profiles and RC Files
- Astronomy for KDE
- Understanding Ceph and Its Place in the Market
- Maru OS Brings Debian to Your Phone
- OpenSwitch Finds a New Home
- Git 2.9 Released
- SoftMaker FreeOffice
- The Giant Zero, Part 0.x
- What's Our Next Fight?
With all the industry talk about the benefits of Linux on Power and all the performance advantages offered by its open architecture, you may be considering a move in that direction. If you are thinking about analytics, big data and cloud computing, you would be right to evaluate Power. The idea of using commodity x86 hardware and replacing it every three years is an outdated cost model. It doesn’t consider the total cost of ownership, and it doesn’t consider the advantage of real processing power, high-availability and multithreading like a demon.
This ebook takes a look at some of the practical applications of the Linux on Power platform and ways you might bring all the performance power of this open architecture to bear for your organization. There are no smoke and mirrors here—just hard, cold, empirical evidence provided by independent sources. I also consider some innovative ways Linux on Power will be used in the future.Get the Guide