HAProxy use server when comes from DOWN to UP

Question

I have a server that runs HAProxy to load balance our mysql servers. Some of the server may go down when we have low average load for a extensive period of time, but, in the future, if the load becomes high again, those servers goes up automatically. The problem is when an instance goes down, HAProxy never looks to it again, so when the instance is up again, it is ignored. To fix this we reboot when needed.

Here is our configuration file:

global
    log 127.0.0.1 local0 notice
    user haproxy
    group haproxy

defaults
    log global
    retries 2
    timeout connect 3000
    timeout server 5000
    timeout client 5000

listen mysql-cluster
        bind 0.0.0.0:3306
        mode tcp
        option mysql-check user haproxy_check
        balance leastconn
        server mysql-1 ********:3306 check
        server mysql-2 ********:3306 check

Maybe if I change the retries from 2 to a big number it could solve our problem?

EDIT As requested, here is my HAProxy version:

$ haproxy -v
HA-Proxy version 1.4.24 2013/06/17
Copyright 2000-2013 Willy Tarreau <[email protected]>

Thanks

Changing retries will not change anything. That parameter refers to retries when establishing a backend connection attempt fails on a backend that HAProxy thought was healthy, even though it isn't, which is rare. You're saying no attempts are made, so that isn't applicable. When you say you have to reboot... you have to reboot what? Also, is ********:3306 an IP address or a hostname? — Michael - sqlbot
I need to reboot HAProxy, so it loads the config file again and tries to establish a connection to both servers. ********:3306 is a hostname. — Maurício Giordano
I think I know what the issue will be. Please mention your HAProxy version. (haproxy -v) — Michael - sqlbot

Tim Bikbaev Tim Bikbaev · Accepted Answer · 2017-06-12T07:14:03

We had similar situation in our AWS infrastructure: HAProxy on every instance for RDS replicas access (we have 3 replicas, but application able to work only with one hostname). We solve this issue globally by replacing HAProxy with Route53 internal using with multiple records with same name ( for example db.example.internal) and same weight (Weighted Route53 policy). Also we create Route53 health check for every replica (TCP 3306 port check). For us this solution works great and if we need to add/delete RDS replica - the only place where we need to change - Route53 record and health check (we don't need to maintain every instance HAProxy status/conf)

HAProxy use server when comes from DOWN to UP

1 Answers