Why does Elastic Load Balancing report 'Out of Service'?

Question

I am trying to set up Elastic Load Balancing (ELB) in AWS to split the requests between multiple instances. I have created several images of my webserver based on the same AMI, and I am able to ssh into each individually and access the site via each distinct public DNS.

I have added each of my instances to the load balancer, but they all come back with the Status: Out of Service because they failed the health check. I'm mostly confused because I can access each instance from its public DNS, but I get a timeout whenever I visit the load balancer DNS name.

I've been trying to read through all the docs and googling it, but I'm stuck. Any pointers or links in the right direction would be greatly appreciated.

Answer 1

I contacted AWS support about this same issue. Apparently their system doesn't know how to handle cases were all of the instances behind the ELB are stopped for an extended amount of time. AWS support can manually refresh the statuses, if you need them up immediately.

The suggested fix it to de-register the ec2 instances from the ELB instead of just stopping them and re-register them when you start again.

Answer 2

Health check is (by default) made by accessing index.html on each instance incorporated in load balancer. If you don't have index.html in document root of instance - default health check will fail. You can set custom protocol, port and path for health check when creating elastic load balancer.

Answer 3

Finally I got this working. The issue was with the Amazon Security Groups, because I've restricted the access to port 80 to few machines on my development area and the load balancer could not access the apache server on the instance. Once the load balancer gained access to my instance, it gets In Service.

I checked it with tail -f /var/log/apache2/access.log in my instance, to verify if the load balancer was trying to access my server, and to see the answer the server is giving to the load balancer.

Hope this helps.

Answer 4

If your web server is running fine, then it means the health check goes on a url that doesn't return 200.

A trick that works for me : go on the instance, type curl localhost:80/pathofyourhealthcheckurl

After you can adapt your health check url to always have a 200 response.

Answer 5

In my case, the rules on security groups assigned to the instance and the load balancer were not allowing traffic to pass between the two. This caused the health check to fail.

Answer 6

I to faced same issue , i changed Ping Protocol from https to ssl .. it worked !

Go to Health Check  --> click on Edit Health Check -- > change Ping protocol from HTTPS to SSL

Ping Target SSL:443
Timeout 5 seconds
Interval    30 seconds
Unhealthy Threshold 5
Healthy Threshold   10

Answer 7

I would like to provide you a general way to solve this problem. When you have set up you web server like apache or nginx, try to read the access log file to see what happened. In my occasion, it report 401 error because I have add the basic auth in nginx. Of course, just like @ivankoni remind, it may because of the document you check is not exist.

Answer 8

I was working on the AWS Tutorial on hosting a web app and ran into this problem. Step 7b states the following:

"Set Ping Path to /. This sends queries to your default page, whether it is named index.html or something else."

They could have put the forward slash in quotations like this "/". Make sure you have that in your health checks and not this "/." .

Answer 9

Adding this because I've spent hours trying to figure it out...

If you configured your health check endpoint but it still says Out of Service, it might be because your server is redirecting the request (i.e. returning a 301 or 302 response).

For example, if your endpoint is supposed to be /app/health/ but you only enter /app/health (no trailing slash) into the health check endpoint field on your ELB, you will not get a 200 response, so the health check will fail.

Answer 10

I had a similar issue. The problem appears to have been caused due to my using a HTTP health check and also using .htaccess to password protect the site.

Answer 11

I got the same error, in my case had to copy the particular html file from s3 bucket to "/var/www/html" location. The same html referenced in load balancer path.

The issue resolved after copying html file.

Answer 12

For anyone else that sees this thread as this isn't listed:

Check that the health check is checking the port that the responding server is listening on.

E.g. node.js running on port 3000 -> Point healthcheck to port 3000;

Not port 80 or 443. Those are what your ALB will be using.

I spent a morning on this. Yes.

Answer 13

I had this issue too, and it was due to both my inbound and outbound rule for the Load Balancer's Security Group only allowing HTTP traffic on port 80. I needed to add another rule for HTTPS traffic on port 443.