114
votes

I'm trying to speed up my benchmark (3 tier web architecture), and I have some general questions related to Memcache(d) and Varnish.

  • What is the difference?
    It seems to me that Varnish is behind the web server, caching web pages and doesn't require change in code, just configuration.
    On the other side, Memcached is general purpose caching system and mostly used to cache result from database and does require change in get method (first cache lookup).

  • Can I use both? Varnish in front web server and Memcached for database caching?

  • What is a better option?

    (scenario 1 - mostly write,
    scenario 2 - mostly read,
    scenario 3 - read and write are similar)

2

2 Answers

276
votes
  • Varnish is in front of the webserver; it works as a reverse http proxy that caches.
  • You can use both.
  • Mostly write -- Varnish will need to have affected pages purged. This will result in an overhead and little benefit for modified pages.
  • Mostly read -- Varnish will probably cover most of it.
  • Similar read & write -- Varnish will serve a lot of the pages for you, Memcache will provide info for pages that have a mixture of known and new data allowing you to generate pages faster.

An example that could apply to stackoverflow.com: adding this comment invalidated the page cache, so this page would have to be cleared from Varnish (and also my profile page, which probably isn't worth caching to begin with. Remembering to invalidate all affected pages may be a bit of an issue). All the comments, however, are still in Memcache, so the database only has to write this comment. Nothing else needs to be done by the database to generate the page. All the comments are pulled by Memcache, and the page is recached until somebody affects it again (perhaps by voting my answer up). Again, the database writes the vote, all other data is pulled from Memcache, and life is fast.

Memcache saves your DB from doing a lot of read work, Varnish saves your dynamic web server from CPU load by making you generate pages less frequently (and lightens the db load a bit as well if not for Memcache).

33
votes

My experience comes from using Varnish with Drupal. In as simple terms as possible, here's how I'd answer:

In general, Varnish works for unauthenticated (via cookie) traffic and memcached will cache authenticated traffic.

So use both.