AWS ALB returns 502 and request doesnt reach targets

Question

We have an ALB load balancing 2 targets. But we are experiencing 502 responses from ALB quite often, that interrupts UI to throw an error. Our application is UI based running on node js and uses socket.io.

Sample log from ALB access log.

https 2019-06-10T09:29:46.987095Z app/DES-G1-USE4-ext-elb/7c8fddfc050d66f6 184.51.199.55:55418 10.72.72.155:8888 0.000 8.697 -1 502 - 876 485 "GET https://designer-use4.genesyscloud.com:443/socket.io/?EIO=3&transport=polling&t=Mj0kC0w HTTP/1.1" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:65.0) Gecko/20100101 Firefox/65.0" ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 arn:aws:elasticloadbalancing:us-east-1:361190373704:targetgroup/DES-G1-USE4-tg-desg/4cc8feabb9ee8f89 "Root=1-5cfe2302-b19c6d5059c2fc6096e048e0" "-" "session-reused" 0 2019-06-10T09:29:38.289000Z "forward" "-" "-"

Here -1 502 - means that request is forwarded to the backend by ELB, but the target is not responded. And the connection is b/w ELB and target is closed somehow. As per https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-access-logs.html.

In our application log, there is no trace of this request reached our application.

Please help us to debug this issue from ALB.

Answer 1

Make sure that your application is responding with a 200-399 response code on the / path. The behavior that you are mentioning is often because the Load Balancer Health Check is not able to verify that your application is running correctly. Also, make sure that your server is able to respond on HTTP that path ( HTTP/2 is not yet supported ).

Answer 2

Potential causes of this:

• firewall: make sure security groups of your application server and ELB are able to connect to each other on the designated port.

• health check: the load balancer will kick an application server that crashes a few times. Make sure that the application server is attached and showing as healthy on the dashboard.