Archiving and Compression

Chapter 8 from Scott Granneman's new book "Linux Phrasebook", the pocket guide every linux user needs. Linux Phrasebook offers a concise reference that, like a language phrasebook, can be used "in the street." The book goes straight to practical Linux uses, providing immediate solutions for day-to-day tasks.
Chapter 8: Archiving and Compression

Although the differences are sometimes made opaque in casual conversation, there is in fact a complete difference between archiving files and compressing them. Archiving means that you take 10 files and combine them into one file, with no difference in size. If you start with 10 100KB files and archive them, the resulting single file is 1000KB. On the other hand, if you compress those 10 files, you might find that the resulting files range from only a few kilobytes to close to the original size of 100KB, depending upon the original file type.

Note - In fact, you might end up with a bigger file during compression! If the file is already compressed, compressing it again adds extra overhead, resulting in a slightly bigger file.

All of the archive and compression formats in this chapter — zip, gzip, bzip2, and tar — are popular, but zip is probably the world's most widely used format. That's because of its almost universal use on Windows, but zip and unzip are well supported among all major (and most minor) operating systems, so things compressed using zip also work on Linux and Mac OS. If you're sending archives out to users and you don't know which operating systems they're using, zip is a safe choice to make.

gzip was designed as an open-source replacement for an older Unix program, compress. It's found on virtually every Unix-based system in the world, including Linux and Mac OS X, but it is much less common on Windows. If you're sending files back and forth to users of Unix-based machines, gzip is a safe choice.

The bzip2 command is the new kid on the block. Designed to supersede gzip, bzip2 creates smaller files, but at the cost of speed. That said, computers are so fast nowadays that most users won't notice much of a difference between the times it takes gzip or bzip2 to compress a group of files.

Note - Linux Magazine published a good article comparing several different compression formats, which you can find at

zip, gzip, and bzip2 are focused on compression (although zip also archives). The tar command does one thing — archive — and it has been doing it for a long time. It's found almost solely on Unix-based machines. You'll definitely run into tar files (also called tarballs) if you download source code, but almost every Linux user can expect to encounter a tarball some time in his career.

Archive and Compress Files Using zip


zip both archives and compresses files, thus making it great for sending multiple files as email attachments, backing up items, or for saving disk space. Using it is simple. Let's say you want to send a TIFF to someone via email. A TIFF image is uncompressed, so it tends to be pretty large. Zipping it up should help make the email attachment a bit smaller.

Note - When using ls -l, I'm only showing the information needed for each example.

   $ ls -lh
   -rw-r--r-- scott scott 1006K young_edgar_scott.tif
   $ zip young_edgar_scott.tif
   adding: young_edgar_scott.tif (deflated 19%)
   $ ls -lh
   -rw-r--r-- scott scott 1006K young_edgar_scott.tif
   -rw-r--r-- scott scott 819K

In this case, you shaved off about 200KB on the resulting zip file, or 19%, as zip helpfully informs you. Not bad. You can do the same thing for several images.

   $ ls -l
   -rw-r--r-- scott scott 251980 edgar_intl_shoe.tif
   -rw-r--r-- scott scott 1130922 edgar_baby.tif
   -rw-r--r-- scott scott 1029224 young_edgar_scott.tif
   $ zip edgar_intl_shoe.tif edgar_baby.tif young_edgar_scott.tif
   adding: edgar_intl_shoe.tif (deflated 4%)
   adding: edgar_baby.tif (deflated 12%)
   adding: young_edgar_scott.tif (deflated 19%)
   $ ls -l
   -rw-r--r-- scott scott 251980 edgar_intl_shoe.tif
   -rw-r--r-- scott scott 1130922 edgar_baby.tif
   -rw-r--r-- scott scott 2074296
   -rw-r--r-- scott scott 1029224 young_edgar_scott.tif

It's not too polite, however, to zip up individual files this way. For three files, it's not so bad. The recipient will unzip and end up with three individual files. If the payload was 50 files, however, the user would end up with files strewn everywhere. Better to zip up a directory containing those 50 files so when the user unzips it, he's left with a tidy directory instead.

   $ ls -lF 
   drwxr-xr-x scott scott edgar_scott/
   $ zip edgar_scott
   adding: edgar_scott/ (stored 0%)
   adding: edgar_scott/edgar_baby.tif (deflated 12%)
   adding: edgar_scott/young_edgar_scott.tif (deflated 19%)
   adding: edgar_scott/edgar_intl_shoe.tif (deflated 4%)
   $ ls -lF
   drwxr-xr-x scott scott   160 edgar_scott/
   -rw-r--r-- scott scott 2074502

Whether you're zipping up a file, several files, or a directory, the pattern is the same: the zip command, followed by the name of the Zip file you're creating, and finished with the item(s) you're adding to the Zip file.



Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Unzipping Password Protected Zips

Anonymous's picture

You left out how to unzip ZIP files that are password protected in Linux. I'm searching for this elusive bit of information on the internet right now...

Password protectedly adding files by PHP code was not found

Farrukh Shahzad's picture

Password protectedly adding files by PHP code was not found on the internet when i was searching for it... so i come across your article and it gave me the idea to why not issue a system command by php to add files in zip and even protect the files by password ;)


Amelia's picture

RAR is good and free too. It supports passwords and can make SFX archives.

No mention of lzma?

Brian Cain's picture

How about rzip or lzma? I recall an article in the print edition within the last ten or eleven issues that compared the cpu overhead of each compression method against compression ratios (and possibly other parameters). Anyways, rzip is memory and cpu intensive, IIRC, but has the potential to make enormous savings. I think it's the same as burrows-wheeler over larger data sets, possibly. Worthwhile for stuff that won't be frequently decompressed, IMO.


Anonymous's picture

actually rzip levels are in search buffer sizes:

-0 = 100MB
-1 = 100MB
-x = x00MB for x>0 and x<=9

cpu intensive? well depends. I hacked bzip2 compression hooks out of the rzip and it's one of the fastest pre archiving filters with best compression ratio for mysql dump of dbmail database.

yup found bug but only in decompression algorithm - not the data itself. yes - made Andrew to fix it.

Correction to wording

DAKH's picture


In the section "Archive Files with tar", paragraph 3, you state that tar is "designed to compress entire directory structures". I think this should read "designed to archive...", since this section deals only with tar's standalone use as an archival tool and since this article/chapter is intended to highlight the difference between archiving and compressing. Other than that, this is a very handy primer on archiving and compressing in *nix.

bzip2 -9

Chris Thompson's picture

The article states that the default block size for bzip2 is -6. The man page for my system (Ubuntu 6.06) states that -9 is the default, and I am unaware of any system where -6 is the default.


TROGDOR's picture

Making -9 the default

Craig Buchek's picture

An easier way to default to the best (-9) compression level would be to export GZIP='-9' and ZIPOPTS='-9' into your environment.

Geek Guide
The DevOps Toolbox

Tools and Technologies for Scale and Reliability
by Linux Journal Editor Bill Childers

Get your free copy today

Sponsored by IBM

Upcoming Webinar
8 Signs You're Beyond Cron

Scheduling Crontabs With an Enterprise Scheduler
11am CDT, April 29th
Moderated by Linux Journal Contributor Mike Diehl

Sign up now

Sponsored by Skybot