Apache HttpClient loadbalancing pooled connections

12
votes

We are using persistent connections, and have tried forcing connections to get dropped after x amount of time. While I see we can in theory use ConnectionKeepAliveStrategy, form what I can tell this only applies after a response..i.e. while the connection is idle.

The issue we are having..

Assume 1 client, hitting 2 servers (A,B) via a loadbalancer. When one of the servers goes offline (B), all the new connections are established on the against the server (A). Now when the the other server (B) comes back online, it will remain idle as all the connections are on the other server (A). As long the client keeps accessing a connection below the idle timeout/keepalive, this will continue, leaving the B server idle (aka with zero connections).

What we want to do..is force the all persistent connections to periodically get closed out (within a 'randomized time window'. Ideally we do not want all connections to reset at the same time). Any suggestions on doing this?

We tried extending HttpClientConnectionManager, and tracking how long a connection had been open for, then close it out after x amount time...however this does not seem to work. I'm guessing this is because HttpClientConnection is not in fact the actual connection, but is rather a proxy and looks like underneath this proxy, it is actually 'using' one of the established connections, as such making its impossible to actually track the time those underlying connections have been established for.

Thoughts?

Right now I'm toying with idea of simply calling: HttpRequestBase.abort() on 1 connection per minute once we have executed a request on it, which I think would get us some what closer to desired behavior.

2
javaapache-httpclient-4.x
Are you in control of the load balancer? Maybe one solution is to turn off sticky sessions. - kuporific
There are no sticky sessions, its round-robbin..but once the persistent connection is created..it stays..hence the need to close the persistent connections periodically, so the lb can once again loadbalance across the servers. - danomano
Did you have any break through in this scenario? I exactly have the same scenario looking for solid ideas on how to deal with it. - jagamot
Well we ended up periodically (2min) sending a bogus request to the server to "/", and abort that connection upon its response, this ensures that at least every 2 minutes one of the persistent connections will reconnect to the lb, forcing connections to rebalance over time. (even thought we should reconfigure the lb to use least load to ensure it does the right thing). - danomano

2 Answers

4
votes

One can limit the total life time of a connection by using TTL (time to live) parameter.

HttpClientBuilder.create()
        .setConnectionTimeToLive(1, TimeUnit.MINUTES)
        .build();

That will force all connections to get renewed after one minute.

4
votes

I don't think you're using the load balancer properly. If you're wired like this:

              +--> SA
C <---> LB <--+
              +--> SB

The client can have persistent connections to the load balancer. The LB<-->SA and LB<-->SB connections can be with persistent connections or not, doesn't matter. The Load balancer should understand HTTP and route on that layer, and not just TCP connections. Thus two incoming (to the LB) HTTP requests on the same persistent connection can be routed to two separate servers.