Varnish client.identity -- how to split load more evenly?

Question

I have two Varnish servers, in a middle-tier between an F5 load balancer and two application servers (Apache httpd). I want Varnish to split the load more-or-less evenly between the two Apache servers. Here's the important bits of the configuration I'm using:

backend prod1 {
  .host = "172.16.1.1";
  .port = "80";
}

backend prod2 {
  .host = "172.16.2.1";
  .port = "80";
}

director wpprod client {
  { .backend = "prod1"; .weight = 1; }
  { .backend = "prod2"; .weight = 1; }
}

sub vcl_recv {
  set client.identity = req.http.user-agent;
  set req.backend = wpprod;
}

The intent is to take a header that shouldn't change during a session, but should be semi-random (the client's User-Agent: header), and use it to split the load between the two back-end servers.

Unfortunately, in practice, the load is being split about 70-30, with one backend getting roughly twice as much traffic as the other.

I've also tried using the X-Forwarded-For: header that my F5s are adding as the client.identity, but had the same problem.

I could just tweak the weights a bit, to try to even out the load, but that's not really scalable, or feasible with more than a few servers.

Any suggestions on headers I can use that will result in a more even split of traffic?

So, this ultimately turned out not to be a Varnish problem at all. Elsewhere in our network, a runaway search engine/spider was hammering these servers. Since it was about 40% of all of our traffic, and since every request it sent had the same User-Agent:, fixing that problem restored the roughly 50-50 balance. — David E. Smith

Carlos Abalde Carlos Abalde · Accepted Answer · 2016-09-26T14:47:38

Assuming you're using Varnish 3.x and also assuming you need some kind of stickiness here (otherwise you could a random or round-robin director), you could populate client.identity with client.ip, or, even better, you could generate / use your own cookie to assign each client to a group:

...

sub vcl_recv {
    cookie.parse(req.http.Cookie);

    unset req.http.X-Group;
    if (cookie.get("group")) {
        set client.identity = cookie.get("group");
    } else {
        set req.http.X-Group = std.random(1, 1000);
        set client.identity = req.http.X-Group;
    }

    set req.backend = wpprod;
}

sub vcl_deliver {
    if (req.http.X-Group) {
        header.append(
            resp.http.Set-Cookie,
            "group=" + req.http.X-Group + "; domain=.example.com; path=/; expires=" + cookie.format_rfc1123(now, 365d));
    }
}

Varnish client.identity -- how to split load more evenly?

2 Answers