Varnish cache crawler request?

0
votes

I have a question about Varnish. I know that Varnish will cache a web page when a user first time visit a web page and it will serve the cache to next user who visit the page later.

My question is do varnish also cache a web page when a web crawler like GoogleBot come and crawl that web page?

1
varnishvarnish-vcl

1 Answers

1
votes

Yes. Unless configured otherwise, Varnish will treat bot requests the same as any other request.

This can be useful as Googlebot can warm the cache for real users. It also means that Googlebot gets fast server response times, which can be beneficial for your site.