Varnish Cache - Cache a 403 response

Question

We're using varnish in front of a AWS S3 bucket and things have been running really well as we've had a 98.4% hit rate which has saved us from very large S3 bills!

Our applications now need to be able to make requests for files which may or may not exist yet. When this happens Varnish will make a request to S3 and receive a 403 (permission denied) response. We catch that response in the vcl_error function as it allows us to display a custom error message. Since we're expecting 400-500 requests per second with about 40% being for files which don't exist yet we will run into cost issues with S3.

My question is, is it possible to have Varnish remember that the file returned a 403 and return a cached response? I would like Varnish to wait 5 minutes before requesting the file from the backend. We're running Varnish 3.

I've read the documentation which appears to sugest I can use "set obj.ttl = 5m;" in the vcl_error function but this doesn't seem to work.

Thanks! Alan

nonamenix nonamenix · Accepted Answer · 2015-05-22T19:32:48

Yes, you can cache it. Just check status code of response from S3 and set ttl.

Varnish 3:

sub vcl_fetch {
    if (beresp.status == 403 || beresp.status == 404 || beresp.status >= 500)
    {
        set beresp.ttl = 3s;
    }
}

Varnish 4:

sub vcl_backend_response {
    if (beresp.status == 403 || beresp.status == 404 || beresp.status >= 500)
    {
        set beresp.ttl = 3s;
    }
}

Varnish Cache - Cache a 403 response

1 Answers