ELB target group - target is not added automatically when the a task is is launched by ECS

Question

I have a UAT ECS cluster with two EC2 instances. I have one of the services (inside the same cluster) which has Desired Task = 1 (so it runs on any one of the two EC2 instances). I have a ELB and targetgroup having the healthy instance.

Problem : Whenever a deployment happens, the running task is stopped and the new task is started in the other EC2 instance. This does not update the target group accordingly. So every time after the deployment, the target group holds the previous unhealthy EC2 target, where the task was running before the deployment.

Is this an expected behavior from AWS ? Any suggestion how I can bypass the manual action I take every time after deployment?

Answer 1

It's definitely not expected behaviour. Do you have the load balancing set up on the ECS service?

For instance, if I use the CLI to describe a service:

$ aws ecs describe-services --services <serviceName> --cluster <clusterName>
{
    "services": [
        // a bunch of stuff
        "loadBalancers": [
            {
                "containerName": <container name>,
                "targetGroupArn": <ELB Target Group ARN>
                "containerPort": <container port number>
            }
        ]
        // more stuff here
    ]
}

With this in place, every time the ECS launches a new task or removes an existing one, the load balancer target group is updated.

If you didn’t configure the load balancer when you made the service, unfortunately you can’t update that part of the service after the fact. All you can do is create a new service and remove the old one.