How to scale in/out EC2 instances based on ECS cluster resources availability?

4
votes

I have multiple services running in my ECS cluster. Each service contains one or more tasks based on CPU utilization or a number of users. I have deployed these containers with EC2 launch type. Now, I want to increase/decrease the number of EC2 instances based on available resources in the cluster. Let's say there are four ECS tasks running in two m5.large instances.

Now, if an ECS service increases the number of tasks and there aren't enough resources available in the cluster, how can I spin up an instance and add to the cluster?

And same goes for vice versa. If there is instance running with no ecs task in it, how can I destroy it automatically?

PS - I was using Fargate. Since it's cost is very high, I moved to EC2 instances.

1
amazon-web-servicesamazon-ec2containersamazon-ecs
You need to use ASGs to control the EC2 instances in your ECS cluster. You can scale these up/down based on CPU, memory, etc. – dmulter
I don't want to scale up/down based on CPU or memory. Because there are some cases when CPU utilization or memory usage is not much high still i need to increase the number of tasks in a service. – Nitesh

1 Answers

1
votes

you need to setup your ecs cluster instances in a ASG as @Nitesh says, second you need to set up a cloudwatch alert based in a key metric, with ecs is complex because you need to set up two autoscaling policies one by service another one to scale up your instances, for ec2 the metric that you could use is Cluster CPU reservation and /or Cluster memory reservation.

The scheme works like this your service increases the number of the desired container by an autoscaling rule using a key metric for your service as could be de CPU usage or the number the request in a load balancer and in consequence the Cluster CPU reservation increase this triggers the cloudwatch alert and your ASG increase the number of instaces.

Some tips scale up fast, and scale down slow this could by handle by setting up the time of the alerts

For the containers use Service Auto Scaling and Target tracking policies for more info see

https://docs.aws.amazon.com/AmazonECS/latest/developerguide/cloudwatch-metrics.html#cluster_reservation

https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-auto-scaling.html

https://aws.amazon.com/blogs/compute/automatic-scaling-with-amazon-ecs/

I hope this help

Regards