Amazon multiple ELB instances using same Auto scaling group

Question

I have 2 Amazon ELB instances using the same Auto scaling group.

I have the Health Check type set to ELB for the Auto scaling group.

On each EC2 instance from the ASG I have 2 web applications (they are hosted in IIS, but I don't think it's really relevant).

Once I have every thing up and running I shut down the IIS on one of those instances so that the health checks from both ELB fail for that specific EC2 instances. (The ASG still sees the instances as healthy, but the ELB sees them as out of service).

My ASG has

Desired instances: 2
Min: 2
Max: 4

I was expecting that the EC2 instance be terminated and the ASG create a new one, but this doesn't happen.

Any ideas ? Thanks

PS: It works fine for 1 ELB instance and 1 ASG with only one web application under IIS.

Looks like a corner case. Is there any reason why you two web applications on the same server? — Rico

Tom Feiner Tom Feiner · Accepted Answer · 2014-04-23T20:09:47

From the documentation:

If there are multiple load balancers associated with your Auto Scaling group, Auto Scaling will check the health state of your EC2 instances by making health check calls to each load balancer. For each call, if the Elastic Load Balancing action returns any state other than InService, the instance will be marked as unhealthy. After Auto Scaling marks an instance as unhealthy, it will remain in that state, even if subsequent calls from other load balancers return an InService state for the same instance.

They do not mention the termination actions spesificaly, but from what I understand, this will set the instance as unhealthy in the spesific ELB where it fails the healthchecks - but as the other ELB is marking it as InService, there is no reason for termination.

Ofcourse, this being AWS, you can always use the API to write some code to detect this, and terminate (set unhealthy) the problematic instance which works for one ELB and not the other.

Or a cleaner approach would be to split this to separate ASGs each with one ELB.

Amazon multiple ELB instances using same Auto scaling group

2 Answers