"Nothing travels faster than the speed of light, with the possible exception of
bad news, which obeys its own set of laws" ~ Douglas Adams

Eh, what can I say? I enjoy a good quote, especially when it involves the topics that I discuss in my blog posts.  Have you ever had those moments where you just need a file from a website and you need it now?  Maybe it's the latest patch to some software, or a new version of your favorite distro that doesn't offer bit-torrent downloads.  Nothing is worse than that moment when your heart sinks as you watch your download rate drop to unbearable numbers.  When you realize that it would probably be quicker to write out a bunch of 1s and 0s than it would to finally get the file you're waiting on.

Well, I know I've had this problem many times before.  But about 6 months ago while sitting in #linuxjournal's IRC channel, I was expressing my *ahem* frustrations about my internet provider's download speed while fetching the latest VirtualBox download.  Just about the point where I was going to give up, nerdboy4200 (Josh) recommended the Axel download accelerator application. As I had never heard of Axel, and a tad suspicious of something that claimed to be a 'multithread' download accelerator, I decided to bite the bullet and give it a
try.  I'll tell you what, boy was I surprised!  Now I'm not one to just put my faith in 'oh, that was a quick download' (If you followed my personal blog post on iperf, I'm a stickler for hard data) so I did some comparisons and number crunching using various websites at different points in the day.  Needless to say, I'm still impressed.  So thanks to nerdboy4200 a blog post was born on the Axel download accelerator.

So just what is Axel? The developers call it a download accelerator but if you want to get technical, it can pull in multiple http or ftp streams into a single download location.  In a nutshell, it's wget hopped up on espresso. Wget is a single stream download in that you point it to a single download location and it fetches the file as requested.  With Axel, you can tell the
application to point to multiple ftp mirrors, or a single http download location, but pull multiple streams at the same time, thus increasing your download speed.

Why would you use Axel? Well as I outlined earlier, it's the perfect utility for those connections to wherever you are downloading from that may have a limit on ftp stream speeds, http sessions that burst and slow down, or servers with a slow connection.  As you will see later on in this post, the hard data was actually quite unbelievable, and I was truly happy with the results.

Now of course, for every positive there comes a possible negative.  If you're using a multistream download utility on a network keep in mind that you will be opening x times as many pipes at up to x times the download rate that you would usually get.  For instance, if you are downloading a single stream at 800 Kb/s, and you want to download 2 streams, now you are sucking down the bandwidth on your network connection for 2 simultaneous streams running 800 Kb/s or greater. Keep this in mind if you are on someone else's network as a guest (or employee) as this may cut into their network throughput.  I somehow doubt your significant other will take kindly to buffering the latest episode of their TV show online while you eat up bandwidth from kernel.org.  However, Axel does allow you to set the number of streams and maximum amount of bandwidth to the download stream, which I will cover here shortly.

The other thing to watch out for is that of common courtesy.  If you're downloading from an ftp site where a lot of other people will be downloading from (say kernel.org), the best thing you can do is utilize Axel's ability to multistream from multiple mirrors instead of a single ftp location.  This way you're not using up the bandwidth from a single mirror, but spreading the bandwidth out among multiple ftp mirrors.

Axel Development and Download

Before you go commenting about how Axel is dead, I'll give you that one. Active development on Axel seems to have come to a standstill in 2009. But the beauty of Open Source software means that there's always someone willing to see the diamond in the rough by forking such an awesome project.  And someone has done such a thing with Axel.  Another thing about Axel is that Debian based distros should have Axel in their repo already, so it should be as simple
as:

apt-get install axel

For the rest of us, there's still .tar.gz packages of which I will get into here shortly.

Axel's main website can be found here:
http://axel.alioth.debian.org/

Their Freshmeat.net page can be found here:
http://freshmeat.net/projects/axel/

And the fork for Axel (via GitHub) can be found here:
https://github.com/emiraga/axel

The person that has been updating the Axel codebase via GitHub has done a pretty good job of keeping up-to-date with his releases.  As of right now the latest release is Jan of 2011.  I personally haven't had any problems with the .tar.gz or the Debian package of Axel as of yet.  But if you want to put on your big britches and play with GitHub code, you're more than welcome to it.

First things first, time to get Axel installed.  If you're on a Debian distro (be it Debian, Ubuntu, or anything that uses Debian codebase and apt-get package manager) you should be able to just run:

apt-get install axel

If you're using an RPM package manager (such as Fedora 15) check yum as it should be available to you via: 

yum install axel
As I test using Debian, Fedora, and some custom distros I tend to install packages from source (tar.gz) and go from there.  So for the rest of you, cd to your favorite download directory and let's get to work.

cd ~/Download wget
http://alioth.debian.org/frs/download.php/3015/axel-2.4.tar.gz tar zxvf
axel-2.4.tar.gz && cd axel-2.4 ./configure && make sudo make install

Tada! Now you should be able to type: axel from the commandline. According to Axel's website, it doesn't have any dependencies (granted, you're going to need GCC and Makeutils to actually compile and link the package) such as external libraries.  (So in a perfect world typing Axel should get you the syntax for running the utility).

Command-Line Switches

There really isn't much to Axel when it comes to command-line switches.  If you want more information on the switches you can: man axel, or type axel --help from the command-line to get information on what each switch does. I'll cover the basics here, and also what I have found useful over the short time that I have been using Axel.

-s x    maximum speed (BPS) - Set the maximum speed allowed to download in Bytes per Second, useful when your on a limited network connection, or a guest on someone else's.

-n x    maximum number of connections - Set the maximum number of connections, I tend to max this out around 10 just to be on the safe side

-o f    Output file    - useful if you want to change thisis-my-we-ird.23.2.43.tar.gz file into thisismyweird-232-43.tar.gz

-v    Verbose        - Self explanatory, the more vs you put in here, the more verbose it gets.

-a    alternate indicator    - changes the download indicator I personally enjoy the look of -a over the default. 

Examples:

# Downloads patch with max number of connections set to 10
axel -n 10 http://www.kernel.org/pub/linux/kernel/v2.6/patch-2.6.39.gz

# Downloads patch at 204,800 Bytes/second (200KB/s) and Max 10 connections
axel -s 204800 -n 10 http://www.kernel.org/pub/linux/kernel/v2.6/patch-2.6.39.gz

# Downloads patch from 4 ftp mirrors
axel ftp://ftp.{us,uk,br,fr}.kernel.org/pub/linux/kernel/v2.6/patch-2.6.39.gz

Real life testing and results

No experiment is complete without blowing things up!  Aw, that's what I get for a Mythbusters marathon weekend.  But, alas, no blowing things up with Axel. In this case no experiment is complete without some hard results.  I'm a sucker for fast results, but I also like to see hard data.  For testing purposes I did the following:

1. I downloaded from 2 websites: virtualbox.org and kernel.org

2.virtualbox.org was through http, kernel.org was http and ftp

3. I downloaded a file 5x with wget, 5x with axel -n 10, and 5x with ftp mirrors

4. I ran these tests in the morning, afternoon and evening. 

5. Once the results were in, I took an average from the 5 results for morning, afternoon, and evening and came up with my numbers.

This way, someone can't come back and say 'Well, your cable connection is faster during xx time' or 'maybe you just got lucky and downloaded it when no one else was downloading it'.  I tried to take every possible variable I could and plug it in.  No I didn't stand on my head, drink coffee through a straw and push 'Enter' with my tongue.  Somehow I don't think that would have changed the results.

WGET Test Results:

virtualbox.org: 1.49 Mb/s
kernel.org HTTP: 1.30 Mb/s
kernel.org FTP: 281.5 Kb/s

AXEL Test Results:

virtualbox.org -n 10: 2.57 Mb/s
kernel.org http -n 10: 2.24 Mb/s
kernel.org 4 ftp mirrors: 560.80 Kb/s

Axel comparison:
virtualbox.org: Increase of 1.08 Mb/s = 72% Increase
kernel.org http: Increase of 962.56 Kb/s = 72% Increase
kernel.org FTP: Increase of 279.30 Kb/s = 99% Increase

All I can say is wow. I honestly didn't think I was going to get results like that when nerdboy4200 (Josh) recommended Axel that day 6 months ago. I was seriously impressed. However, I did notice one thing: if your internet speed maxes out at a certain speed (say 2Mb/s) odds are, you're not going to get 4Mb/s on an Axel download, so don't expect miracles.  I did some further testing on other internet connections and noticed that there were times where the results of wget matched that of Axel, only because the speed of the provider I was pulling from matched the max download speed I was allowed to pull on that internet connection.  In that case, both wget and axels results were similar.

Conclusion

It's just amazing when someone recommends an Open Source product through word of mouth, and it becomes something that you end up using on a daily basis. That's just what makes Open Source so powerful, the ability for people to find a product that changes their sys admin life, and pass that knowledge onto the next person they see struggling.  In this case Axel really takes the cake.

Now if you want to 'accelerate' (pun intended) Axel, there are gui utilities out there (axel-kapt comes to mind from debian repos) and a Google search for 'axel debian apt-get' will turn up some interesting results where people have been able to integrate Axel into their apt-get downloads for an increase of package downloading performance.

So the next time you're in #linuxjournal, feel free to drop a line to nerdboy4200 for recommending such an awesome utility.  Without his experimentation with Axel, I'm sure I would still be downloading the latest update to virtualbox today.