Speed Up Your Web Site with Varnish

Now, let's say you want to add a header to the response called X-Hit. The value should be 1 for a cache hit and 0 for a miss. The easiest way to detect a hit is from within the vcl_hit subroutine. Recall that vcl_hit will be executed only when a cache hit occurs. Ideally, you'd set the response header from within vcl_hit, but looking at Table 1 in this article, you see that neither of the response objects (beresp and resp) are available within vcl_hit. One way around this is to set a temporary header in the request, then later set the response header. Let's take a look at how to solve this.

Adding an X-Hit response header:


sub vcl_hit {
    set req.http.tempheader = "1";
}

sub vcl_miss {
    set req.http.tempheader = "0";
}

sub vcl_deliver {
    set resp.http.X-Hit = "0";
    if (req.http.tempheader) {
        set resp.http.X-Hit = req.http.tempheader;
        unset req.http.tempheader;
    }
}

The code in vcl_hit and vcl_miss is straightforward—set a value in a temporary request header to indicate a cache hit or miss. The interesting bit is in vcl_deliver. First, I set a default value for X-Hit to 0, indicating a miss. Next, I detect whether the request's tempheader was set, and if so, set the response's X-Hit header to match the temporary header set earlier. I then delete the tempheader to keep things tidy, and I'm all done. The reason I chose the vcl_deliver subroutine is because the response object that will be sent back to the client (resp) is available only within vcl_deliver.

Let's explore a similar solution that doesn't work as expected.

Adding an X-Hit response header—the wrong way:


sub vcl_hit {
    set req.http.tempheader = "1";
}

sub vcl_miss {
    set req.http.tempheader = "0";
}

sub vcl_fetch {
    set beresp.http.X-Hit = "0";
    if (req.http.tempheader) {
        set beresp.http.X-Hit = req.http.tempheader;
        unset req.http.tempheader;
    }
}

Notice that within vcl_fetch, I'm now altering the back end's response (beresp), not the final response sent to the client. This code appears to work as expected, but it has a major bug. What happens is that the first request is a miss and fetched from the back end, and that response has X-Hit set to "0" then it's cached. Subsequent requests result in a cache hit and never enter the vcl_fetch subroutine. The result is that all cache hits continue having X-Hit set to "0". These are the types of mistakes to look out for when working with Varnish.

The easiest way to avoid these mistakes is to keep those reference tables handy; remember when each subroutine is executed in Varnish's workflow, and always test the results.

Let's look at a simple way to tell Varnish to cache everything for one hour. This is shown only as an example and isn't recommended for a real server.

Cache all responses for one hour:


sub vcl_recv {
    return (lookup);
}

sub vcl_fetch {
    set beresp.ttl = 1h;
    return (deliver);
}

Here, I'm overriding two default subroutines with my own. If I hadn't returned "deliver" from vcl_fetch, Varnish still would have executed its default vcl_fetch subroutine looking for a return value, and this would not have worked as expected.

Once you get Varnish to implement your caching policy, you should run some benchmarks to see if there is any improvement. The benchmarking tool I use here is the Apache benchmark tool, known as ab. You can install this tool as part of the Apache Web server or as a separate package—depending on your system's package manager. You can read about the various options available to ab in either the man page or at the Apache Web site.

In the benchmark examples below, I have a stock Apache 2.2 installation listening on port 80, and Varnish listening on port 6081. The page I'm testing is a very basic Perl CGI script I wrote that just outputs a one-liner HTML page. It's important to benchmark the same URL against both the Web server and Varnish so you can make a direct comparison. I run the benchmark from the same machine that Apache and Varnish are running on in order to eliminate the network as a factor. The ab options I use are fairly straightforward. Feel free to experiment with different ab options and see what happens.

Let's start with 1000 total requests (-n 1000) and a concurrency of 1 (-c 1).

Benchmarking Apache with ab:


ab -c 1 -n 1000 http://localhost/cgi-bin/test

Figure 4. Output from ab Command (Apache)

Benchmarking Varnish with ab:


ab -c 1 -n 1000 http://localhost:6081/cgi-bin/test

Figure 5. Output from ab Command (Varnish)

As you can see, the ab command provides a lot of useful output. The metrics I'm looking at here are "Time per request" and "Requests per second" (rps). You can see that Apache came in at just over 1ms per request (780 rps), while Varnish came in at 0.1ms (7336 rps)—nearly ten times faster than Apache. This shows that Varnish is faster, at least based on the current setup and isolated testing. It's a good idea to run ab with various options to get a feel for performance—particularly by changing the concurrency values and seeing what impact that has on your system.

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

fight club jacket

fight club jacket's picture

This post is really Beneficial and Enthralling, i will follow you back for more of the information.

How did you get to be this

bridgetclara's picture

How did you get to be this great? Its remarkable to see someone put so much passion into a subject. Im glad I came across this. Im glad I took the time to read on past the first paragraph. Youve got so much to say, so significantly to give. I hope men and women realise this and look into your page. http://www.scarfsmall.com/

One Tail Just Isn't Enough

yulia.syukurista's picture

really thanks for your share

I really hope you visit my web
belanja elektronik di evoelectronic.com AND rental mobil surabaya

This is very much

ryushare rereseller's picture

This is very much interesting. Thanks for sharing this useful information.
Regards..
ryushare reseller,
ryushare paypal,
Đặng Ngân

Pages are not updated quickly

Ravi's picture

I had tried it in past, but the problem was the refreshed pages were not shown instantly. It was taking minutes to refresh the page. So, I decided to get rid off it.

Is there anyway to fix this problem for dynamic pages?

now i knew

ryusharereseller's picture

Thanks for sharing this useful information.
Regards..

SEO Tips

pandalinkbuildings's picture

Here is a rapid suggestion to boost your own WEB OPTIMIZATION search rankings to the web site is actually having a look in the taxonomy web theme as well as making use of your keywords as conditions. This way your articles receives linked together whilst your keywords find propagate from the web site.

Reachli - Amplifying your

auto diagnostic tool's picture

Reachli - Amplifying your content reach online via, in-image, in-content,
in-text and social (relevant ads).

Reachli provides the easiest way to market your visual content online - Social Media Marketing, Advertising,
Marketing Online, Advertising Online, Social Media Analytics - Pinterest,
Facebook, Instagram, Tumblr, Twitter.

great post, verу infoгmative.

laulan's picture

great post, verу infoгmative. I
wοndеr ωhy the other speсialists of this sector ԁo not
understand thiѕ. You should сontinue your writing.
I'm sure, you have a huge readers' basе alrеady!
ZOPO C2 Smartphone

What about sites that customize contents for each user?

Anonymous's picture

Thanks for the article.

> It will consider only caching GET and HEAD requests. It won't cache a request with either a Cookie or Authorization header. It won't cache a response with either a Set-Cookie or Vary header

... which means that tools like Varnish are only useful to speed up sites that don't customize contents on a user-basis. I wonder how those other sites eg. Facebook handle the issue, since each page is different for each user.

This is very much

mathan's picture

This is very much interesting. Thanks for sharing this useful information.
Regards..
Mathan
Cloud computing training in Chennai

The Varnish Book

Rubén Romero's picture

If you are getting started with Varnish, make sure to read the Varnish Book:
https://www.varnish-software.com/static/book/

This book is the material we use for our Varnish training classes and it gives a good introduction on the project history, what it is, how VCL works and how Varnish it is related to HTTP and Cache Invalidation.

Read the Varnish Book

Rubén Romero's picture

If you are getting started with Varnish, make sure to read the Varnish Book:
https://www.varnish-software.com/static/book/

This book is the material we use for our Varnish training classes and it gives a good introduction on the project history, what it is, how VCL works and how Varnish it is related to HTTP and Cache Invalidation.

yea

beba's picture

Is it something like maxcdn ?

Varnish works!

redgooli's picture

Trust me! it has made a huge difference for us

Reply to comment | Linux Journal

take the risk's picture

For most up-to-date news you have to go to see the web and on the web
I found this web site as a finest web site for most recent updates.

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix