General questions

What is Varnish?

Please see ref:tutorial-intro.

How...

How much RAM/Disk do I need for Varnish?

That depends on pretty much everything.

I think our best current guidance is that you go for a cost-effective RAM configuration, something like 1-16GB, and an SSD.

Unless you positively know that you will need it, there is little point in spending a fortune on a hand-sewn motherboard that can fit several TB in special RAM blocks, riveted together by leftover watch-makers in Switzerland.

On the other hand, if you plot your traffic in Gb/s, you probably need all the RAM you can afford/get.

How can I limit Varnish to use less RAM?

You can not. Varnish operates in Virtual Memory and it is up to the kernel to decide which process gets to use how much RAM to map the virtual address-space of the process.

How do I instruct varnish to ignore the query parameters and only cache one instance of an object?

This can be achieved by removing the query parameters using a regexp:

sub vcl_recv {
    set req.url = regsub(req.url, "\?.*", "");
}

How can I force a refresh on a object cached by varnish?

Refreshing is often called purging a document. You can purge at least 2 different ways in Varnish:

  1. Command line

    From the command line you can write:

    ban.url ^/$

    to ban your / document. As you might see ban.url takes an regular expression as its argument. Hence the ^ and $ at the front and end. If the ^ is omitted, all the documents ending in a / in the cache would be deleted.

    So to delete all the documents in the cache, write:

    ban.url .

    at the command line.

  2. HTTP PURGE

    VCL code to allow HTTP PURGE is to be found here. Note that this method does not support wildcard purging.

How can I debug the requests of a single client?

The "varnishlog" utility may produce a horrendous amount of output. To be able debug our own traffic can be useful.

The ReqStart token will include the client IP address. To see log entries matching this, type:

$ varnishlog -c -m ReqStart:192.0.2.123

To see the backend requests generated by a client IP address, we can match on the TxHeader token, since the IP address of the client is included in the X-Forwarded-For header in the request sent to the backend.

At the shell command line, type:

$ varnishlog -b -m TxHeader:192.0.2.123

How can I rewrite URLS before they are sent to the backend?

You can use the regsub() function to do this. Here's an example for zope, to rewrite URLs for the virtualhostmonster:

if (req.http.host ~ "^(www.)?example.com") {
  set req.url = regsub(req.url, "^", "/VirtualHostBase/http/example.com:80/Sites/example.com/VirtualHostRoot");
}

I have a site with many host names, how do I keep them from multiplying the cache?

You can do this by normalizing the Host header for all your host names. Here's a VCL example:

if (req.http.host ~ "^(www.)?example.com") {
  set req.http.host = "example.com";
}

How do I do to alter the request going to the backend? You can use the bereq object for altering requests going to the backend, but you can only 'set' values to it. Therefore use req.url to build the request:

sub vcl_miss {
        set bereq.url = regsub(req.url,"stream/","/");
        return(fetch);
}

How can I customize the error messages that Varnish returns?

A custom error page can be generated by adding a vcl_error to your configuration file. The default error page looks like this:

sub vcl_error {
    set obj.http.Content-Type = "text/html; charset=utf-8";

    synthetic {"
    <?xml version="1.0" encoding="utf-8"?>
    <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
     "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
    <html>
      <head>
        <title>"} + obj.status + " " + obj.response + {"</title>
      </head>
      <body>
      <h1>Error "} + obj.status + " " + obj.response + {"</h1>
      <p>"} + obj.response + {"</p>
        <h3>Guru Meditation:</h3>
        <p>XID: "} + req.xid + {"</p>
        <address><a href="http://www.varnish-cache.org/">Varnish</a></address>
      </body>
     </html>
     "};
    return(deliver);
}

How do I instruct varnish to ignore the query parameters and only cache one instance of an object?

This can be achieved by removing the query parameters using a regexp:

sub vcl_recv {
    set req.url = regsub(req.url, "\?.*", "");
}

Can I...

Can I run Varnish on the same system as Apache?

Yes, and many people do that with good success.

There will be competition for resources, but Apache is not particular good at using RAM effectively and Varnish is, so this synergy usually more than compensates for the competition.

Can I run multiple Varnish on the same system?

Yes, as long as you give them different TCP ports and different -n arguments, you will be fine.

Can I cache multiple virtual hosts with one Varnish?

Yes, that works right out of the box.

Can I see what is cached in Varnish?

That is not possible for several reasons. A command to list all the contents of a Varnish cache with millions of objects would bring your Varnish to a standstill while it traverses the index.

Besides, the output is a lot less useful than you might think.

Can I use Varnish to do HTTPS?

Not at present, and while we keep an eye on this, there are no current plans to add HTTPS support, until we can find a way where it adds significant value, relative to running a stand-alone HTTPS proxy such as nginx or pound.

Can Varnish load balance between multiple backends?

Yes, you need VCL code like this:

    director foobar round-robin {
        { .backend = { .host = "www1.example.com"; .port = "http"; } }
        { .backend = { .host = "www2.example.com"; .port = "http"; } }
    }

    sub vcl_recv {
            set req.backend = foobar;
    }

Please see :ref:`tutorial-advanced_backend_servers-directors_`.

Why ...

Why does it look like Varnish sends all requests to the backend? I thought it was a cache?

Please see ref:tutorial-increasing_your_hitrate.

Why does Varnish require the system to have a C compiler?

The VCL compiler generates C source as output (your config file), and uses the systems C-compiler to compile that into a shared library. If there is no C compiler, Varnish will not work.

Isn't that security problem?

The days when you could prevent people from running non-approved programs by removing the C compiler from your system ended roughly with the VAX 11/780 computer.

Troubleshooting

Why am I getting a cache hit, but a request is still going to my backend?

Varnish has a feature called hit for pass, which is used when Varnish gets a response from the backend and finds out it cannot be cached. In such cases, Varnish will create a cache object that records that fact, so that the next request goes directly to "pass".

Since Varnish bundles multiple requests for the same URL to the backend, a common case where a client will get a hit for pass is:
  • Client 1 requests url /foo
  • Client 2..N request url /foo
  • Varnish tasks a worker to fetch /foo for Client 1
  • Client 2..N are now queued pending response from the worker
  • Worker returns object to varnish which turns out to be non-cacheable.
  • Client 2..N are now given the hit for pass object instructing them to go to the backend

The hit for pass object will stay cached for the duration of its ttl. This means that subsequent clients requesting /foo will be sent straight to the backend as long as the hit for pass object exists. The varnishstat can tell you how many hit for pass objects varnish has served. The default vcl will set ttl for a hit_for_pass object to 120s. But you can override this, using the following logic:

sub vcl_fetch {
  if (!obj.cacheable) {
    # Limit the lifetime of all 'hit for pass' objects to 10 seconds
    obj.ttl = 10s;
    return(hit_for_pass);
  }
}

Table Of Contents

Previous topic

Frequently asked questions

Next topic

HTTP

This Page