Speed Up Your Web Site with Varnish

Varnish is a program that can greatly speed up a Web site while reducing the load on the Web server. According to Varnish's official site, Varnish is a "Web application accelerator also known as a caching HTTP reverse proxy".

When you think about what a Web server does at a high level, it receives HTTP requests and returns HTTP responses. In a perfect world, the server would return a response immediately without having to do any real work. In the real world, however, the server may have to do quite a bit of work before returning a response to the client. Let's first look at how a typical Web server handles this, and then see what Varnish does to improve the situation.

Although every server is different, a typical Web server will go through a potentially long sequence of steps to service each request it receives. It may start by spawning a new process to handle the request. Then, it may have to load script files from disk, launch an interpreter process to interpret and compile those files into bytecode and then execute that bytecode. Executing the code may result in additional work, such as performing expensive database queries and retrieving more files from disk. Multiply this by hundreds or thousands of requests, and you can see how the server quickly can become overloaded, draining system resources trying to fulfill requests. To make matters worse, many of the requests are repeats of recent requests, but the server may not have a way to remember the responses, so it's sentenced to repeating the same painful process from the beginning for each request it encounters.

Things are a little different with Varnish in place. For starters, the request is received by Varnish instead of the Web server. Varnish then will look at what's being requested and forward the request to the Web server (known as a back end to Varnish). The back-end server does its regular work and returns a response to Varnish, which in turn gives the response to the client that sent the original request.

If that's all Varnish did, it wouldn't be much help. What gives us the performance gains is that Varnish can store responses from the back end in its cache for future use. Varnish quickly can serve the next response directly from its cache without placing any needless load on the back-end server. The result is that the load on the back end is reduced significantly, response times improve, and more requests can be served per second. One of the things that makes Varnish so fast is that it keeps its cache completely in memory instead of on disk. This and other optimizations allow Varnish to process requests at blinding speeds. However, because memory typically is more limited than disk, you have to size your Varnish cache properly and take measures not to cache duplicate objects that would waste valuable space.

Let's install Varnish. I'm going to explain how to install it from source, but you can install it using your distribution's package manager. The latest version of Varnish is 3.0.3, and that's the version I work with here. Be aware that the 2.x versions of Varnish have some subtle differences in the configuration syntax that could trip you up. Take a look at the Varnish upgrade page on the Web site for a full list of the changes between versions 2.x and 3.x.

Missing dependencies is one of the most common installation problems. Check the Varnish installation page for the full list of build dependencies.

Run the following commands as root to download and install the latest version of Varnish:


cd /var/tmp
wget http://repo.varnish-cache.org/source/varnish-3.0.3.tar.gz
tar xzf varnish-3.0.3.tar.gz
cd varnish-3.0.3
sh autogen.sh
sh configure
make
make test
make install

Varnish is now installed under the /usr/local directory. The full path to the main binary is /usr/local/sbin/varnishd, and the default configuration file is /usr/local/etc/varnish/default.vcl.

You can start Varnish by running the varnishd binary. Before you can do that though, you have to tell Varnish which back-end server it's caching for. Let's specify the back end in the default.vcl file. Edit the default.vcl file as shown below, substituting the values for those of your Web server:


backend default {
    .host = "127.0.0.1";
    .port = "80";
}

Now you can start Varnish with this command:


/usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl 
 ↪-a :6081 -P /var/run/varnish.pid -s malloc,256m

This will run varnishd as a dæmon and return you to the command prompt. One thing worth pointing out is that varnishd will launch two processes. The first is the manager process, and the second is the child worker process. If the child process dies for whatever reason, the manager process will spawn a new process.

Varnishd Startup Options

The -f option tells Varnish where your configuration file lives.

The -a option is the address:port that Varnish will listen on for incoming HTTP requests from clients.

The -P option is the path to the PID file, which will make it easier to stop Varnish in a few moments.

The -s option configures where the cache is kept. In this case, we're using a 256MB memory-resident cache.

If you installed Varnish from your package manager, it may be running already. In that case, you can stop it first, then use the command above to start it manually. Otherwise, the options it was started with may differ from those in this example. A quick way to see if Varnish is running and what options it was given is with the pgrep command:


/usr/bin/pgrep -lf varnish

Varnish now will relay any requests it receives to the back end you specified, possibly cache the response, and deliver the response back to the client. Let's submit some simple GET requests and see what Varnish does. First, run these two commands on separate terminals:


/usr/local/bin/varnishlog
/usr/local/bin/varnishstat

The following GET command is part of the Perl www library (libwww-perl). I use it so you can see the response headers you get back from Varnish. If you don't have libwww-perl, you could use Firefox with the Live HTTP Headers extension or another tool of your choice:


GET -Used http://localhost:6081/

Figure 1. Varnish Response Headers

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

fight club jacket

fight club jacket's picture

This post is really Beneficial and Enthralling, i will follow you back for more of the information.

How did you get to be this

bridgetclara's picture

How did you get to be this great? Its remarkable to see someone put so much passion into a subject. Im glad I came across this. Im glad I took the time to read on past the first paragraph. Youve got so much to say, so significantly to give. I hope men and women realise this and look into your page. http://www.scarfsmall.com/

One Tail Just Isn't Enough

yulia.syukurista's picture

really thanks for your share

I really hope you visit my web
belanja elektronik di evoelectronic.com AND rental mobil surabaya

This is very much

ryushare rereseller's picture

This is very much interesting. Thanks for sharing this useful information.
Regards..
ryushare reseller,
ryushare paypal,
Đặng Ngân

Pages are not updated quickly

Ravi's picture

I had tried it in past, but the problem was the refreshed pages were not shown instantly. It was taking minutes to refresh the page. So, I decided to get rid off it.

Is there anyway to fix this problem for dynamic pages?

now i knew

ryusharereseller's picture

Thanks for sharing this useful information.
Regards..

SEO Tips

pandalinkbuildings's picture

Here is a rapid suggestion to boost your own WEB OPTIMIZATION search rankings to the web site is actually having a look in the taxonomy web theme as well as making use of your keywords as conditions. This way your articles receives linked together whilst your keywords find propagate from the web site.

Reachli - Amplifying your

auto diagnostic tool's picture

Reachli - Amplifying your content reach online via, in-image, in-content,
in-text and social (relevant ads).

Reachli provides the easiest way to market your visual content online - Social Media Marketing, Advertising,
Marketing Online, Advertising Online, Social Media Analytics - Pinterest,
Facebook, Instagram, Tumblr, Twitter.

great post, verу infoгmative.

laulan's picture

great post, verу infoгmative. I
wοndеr ωhy the other speсialists of this sector ԁo not
understand thiѕ. You should сontinue your writing.
I'm sure, you have a huge readers' basе alrеady!
ZOPO C2 Smartphone

What about sites that customize contents for each user?

Anonymous's picture

Thanks for the article.

> It will consider only caching GET and HEAD requests. It won't cache a request with either a Cookie or Authorization header. It won't cache a response with either a Set-Cookie or Vary header

... which means that tools like Varnish are only useful to speed up sites that don't customize contents on a user-basis. I wonder how those other sites eg. Facebook handle the issue, since each page is different for each user.

This is very much

mathan's picture

This is very much interesting. Thanks for sharing this useful information.
Regards..
Mathan
Cloud computing training in Chennai

The Varnish Book

Rubén Romero's picture

If you are getting started with Varnish, make sure to read the Varnish Book:
https://www.varnish-software.com/static/book/

This book is the material we use for our Varnish training classes and it gives a good introduction on the project history, what it is, how VCL works and how Varnish it is related to HTTP and Cache Invalidation.

Read the Varnish Book

Rubén Romero's picture

If you are getting started with Varnish, make sure to read the Varnish Book:
https://www.varnish-software.com/static/book/

This book is the material we use for our Varnish training classes and it gives a good introduction on the project history, what it is, how VCL works and how Varnish it is related to HTTP and Cache Invalidation.

yea

beba's picture

Is it something like maxcdn ?

Varnish works!

redgooli's picture

Trust me! it has made a huge difference for us

Reply to comment | Linux Journal

take the risk's picture

For most up-to-date news you have to go to see the web and on the web
I found this web site as a finest web site for most recent updates.

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix