I have just set up a server with just varnish installed in front of my backend server, where I have two different django sites, being served through nginx+gunicorn
It seem to work, but I get Header Age = 0, and looking at the documentation, that's not very good.
I want to cache pages for anonymous users, but not for authenticated users or if a user have a cookie called "AUTHENTICATION"
Here is my default.vcl
backend django {
.host = "backend1";
.port = "8080";
}
sub vcl_recv {
# unless sessionid/csrftoken is in the request, don't pass ANY cookies (referral_source, utm, etc)
if (req.request == "GET" && (req.url ~ "^/static" || (req.http.cookie !~ "sessionid" && req.http.cookie !~ "csrftoken" && req.http.cookie !~ "AUTHENTICATION"))) {
remove req.http.Cookie;
}
#normalize accept-encoding to account for different browsers
if (req.http.Accept-Encoding) {
if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
# No point in compressing these
remove req.http.Accept-Encoding;
} elsif (req.http.Accept-Encoding ~ "gzip") {
set req.http.Accept-Encoding = "gzip";
} elsif (req.http.Accept-Encoding ~ "deflate") {
set req.http.Accept-Encoding = "deflate";
} else {
# unknown algorithm
remove req.http.Accept-Encoding;
}
}
}
sub vcl_fetch {
# /static and /media files always cached
if (req.url ~ "^/static" || req.url ~ "^/media") {
unset beresp.http.set-cookie;
return (deliver);
}
# pass through for anything with a session/csrftoken set
if (beresp.http.set-cookie ~ "sessionid" || beresp.http.set-cookie ~ "csrftoken" || beresp.http.set-cookie ~ "AUTHENTICATION") {
return (hit_for_pass);
} else {
return (deliver);
}
}
Could it be that sessionid
is set for every user, even if they are not logged in, and that is preventing Varnish effectively cache pages for anon users?
Edit:
Using isvarnishworking.com this is the output:
HTTP/1.1 200 OK
Server: cloudflare-nginx
Date: Fri, 15 Nov 2013 09:30:20 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Set-Cookie: __cfduid=d281023a84b2e5351d109c1848eeca1601384507820317; expires=Mon, 23-Dec-2019 23:50:00 GMT; path=/; domain=.mydomain.com; HttpOnly
Vary: Cookie
X-Frame-Options: SAMEORIGIN
X-Varnish: 1602772074
Age: 0
Via: 1.1 varnish
CF-RAY: cdaec14fab00412
Content-Encoding: gzip
Edit 2:
My new default.vcl:
backend django {
.host = "backend1";
.port = "8080";
}
sub vcl_recv {
#normalize accept-encoding to account for different browsers
if (req.http.Accept-Encoding) {
if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
# No point in compressing these
remove req.http.Accept-Encoding;
} elsif (req.http.Accept-Encoding ~ "gzip") {
set req.http.Accept-Encoding = "gzip";
} elsif (req.http.Accept-Encoding ~ "deflate") {
set req.http.Accept-Encoding = "deflate";
} else {
# unknown algorithm
remove req.http.Accept-Encoding;
}
}
}
sub vcl_fetch {
if (req.url ~ "^/static" || req.url ~ "^/media") {
unset beresp.http.set-cookie;
}
if (beresp.http.set-cookie !~ "sessionid" && beresp.http.set-cookie !~ "csrftoken" && beresp.http.set-cookie !~ "AUTHENTICATION") {
unset beresp.http.set-cookie;
}
}
Result from isvarnishworking.com
HTTP/1.1 200 OK
Server: cloudflare-nginx
Date: Fri, 15 Nov 2013 12:08:42 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Set-Cookie: __cfduid=d55ea1b56e978cbbf3384d0fa2f21571e1384517322491; expires=Mon, 23-Dec-2019 23:50:00 GMT; path=/; domain=.mydomain.com; HttpOnly
Vary: Cookie
X-Frame-Options: SAMEORIGIN
X-Varnish: 1240916568
Age: 0
Via: 1.1 varnish
CF-RAY: cdbd4119f3b0412
Content-Encoding: gzip
Edit 3:
backend default {
.host = "backend1";
.port = "8080";
}
sub vcl_recv {
# unless sessionid/csrftoken is in the request, don't pass ANY cookies (referral_source, utm, etc)
if (req.request == "GET" && (req.url ~ "^/static" || (req.http.cookie !~ "sessionid" && req.http.cookie !~ "csrftoken" && req.http.cookie !~ "AUTHENTICATION"))) {
remove req.http.Cookie;
}
#normalize accept-encoding to account for different browsers
if (req.http.Accept-Encoding) {
if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
# No point in compressing these
remove req.http.Accept-Encoding;
} elsif (req.http.Accept-Encoding ~ "gzip") {
set req.http.Accept-Encoding = "gzip";
} elsif (req.http.Accept-Encoding ~ "deflate") {
set req.http.Accept-Encoding = "deflate";
} else {
# unknown algorithm
remove req.http.Accept-Encoding;
}
}
}
sub vcl_fetch {
# /static and /media files always cached
if (req.url ~ "^/static" || req.url ~ "^/media") {
unset beresp.http.set-cookie;
}
if (beresp.http.set-cookie !~ "sessionid" && beresp.http.set-cookie !~ "csrftoken" && beresp.http.set-cookie !~ "AUTHENTICATION") {
unset beresp.http.set-cookie;
}
}
My backend response (without varnish in front) is:
GET / HTTP/1.1
Host: www.mydomain.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:25.0) Gecko/20100101 Firefox/25.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: nb-no,nb;q=0.9,no-no;q=0.8,no;q=0.6,nn-no;q=0.5,nn;q=0.4,en-us;q=0.3,en;q=0.1
Accept-Encoding: gzip, deflate
Cookie: __cfduid=d8f496aef561efd7a30c3d9f909a02cf31384507505064; sessionid=twoq45r21gn341545ohubilyp739r42ee; _ga=GA1.2.382479980.1384507508
Connection: keep-alive
HTTP/1.1 200 OK
Server: cloudflare-nginx
Date: Fri, 15 Nov 2013 14:37:53 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Vary: Accept-Language, Cookie
X-Frame-Options: SAMEORIGIN
Content-Language: nb
CF-RAY: cdcae94f68105af
Content-Encoding: gzip