Layer4 "Connection refused" with haproxy

3
votes

I need some advise on how to setup haproxy. I have two web-servers up and running. For testing they run a simple node server on port 8080.

Now on my haproxy server I start haproxy which gives me the following:

$> /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg 
[WARNING] 325/202628 (16) : Server node-backend/server-a is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[WARNING] 325/202631 (16) : Server node-backend/server-b is DOWN, reason: Layer4 timeout, check duration: 2001ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[ALERT] 325/202631 (16) : backend 'node-backend' has no server available!

Just one note: If I do: 

haproxy$> wget server-a:8080

I get the response from the node server.

Here is my haproxy.cfg:

#---------------------------------------------------------------------  
# Global settings
#---------------------------------------------------------------------
global
    log         127.0.0.1 local2

    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy

    stats socket /var/lib/haproxy/stats

#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
    mode                    tcp
    log                     global
    option                  tcplog
    option                  dontlognull
    option http-server-close
#   option forwardfor       except 127.0.0.0/8
    option                  redispatch
    retries                 3
    timeout http-request    10s
    timeout queue           1m
    timeout connect         10s
    timeout client          1m
    timeout server          1m
    timeout http-keep-alive 10s
    timeout check           10s
    maxconn                 3000

#---------------------------------------------------------------------
# main frontend which proxys to the backends
#---------------------------------------------------------------------
frontend  www
    bind                        *:80
    default_backend             node-backend

#---------------------------------------------------------------------
# round robin balancing between the various backends
#--------------------------------------------------------------------
backend node-backend
   balance roundrobin
   mode tcp
   server server-a 172.19.0.2:8080 check
   server server-b 172.19.0.3:8080 check

If I remove the check option it seems to work. Any suggestions how I can fix this checking mechanism of haproxy?

3
haproxy

3 Answers

4
votes

You need to get exact ip address of your server with the help of command

ifconfig

and correct the below address in your haproxy.cfg file:

172.19.0.2:8080
172.19.0.3:8080

or modify line like below

server server-a server-a:8080 check
server server-b server-b:8080 check
2
votes

Remove "mode tcp" and change it to "mode http".
Im just guessing here but i suppose haproxy is doing a tcp check against your web server and the web server can not respond to it.
in "mode http" it checks the web server in http mode and expects a "response 200" for L4 check and expects a string (whatever you defined) as a L7 check

eg. L4

backend node-backend balance roundrobin mode http #(NOT NEEDED IF DEFINED IN DEFAULTS) option httpchk server server-a 172.19.0.2:8080 check server server-b 172.19.0.3:8080 check
eg. L7

backend node-backend balance roundrobin mode http #(NOT NEEDED IF DEFINED IN DEFAULTS) option httpchk get /SOME_URI http-check expect status 200 server server-a 172.19.0.2:8080 check server server-b 172.19.0.3:8080 check

2
votes

Another note related to @basickarl's comment on docker. If you are sending into a docker (docker-compose) instance (namely where you have multiple instances of service running) you likely need to define the docker resolver and use it for dns resolution on your backend:

resolver:

resolvers docker_resolver
nameserver dns 127.0.0.11:53

backend usage of resolver:

backend main
    balance roundrobin
    option http-keep-alive
    server haproxyapp app:80 check inter 10s resolvers docker_resolver resolve-prefer ipv4