Restrict services/apps to run in a particular Amazon ECS instance

7
votes

I’m in the process of migrating some workloads to Amazon ECS Container Service and I found a blocker that is preventing me from moving forward.

Ideally, containerized apps running on ECS should be stateless so that they can run in any cluster instance and scale out. But the reality is that I have some apps that are dependent on EBS data volumes. As far as I know, these volumes have to be manually attached to a particular cluster instance and ECS apps should be placed in that instance to be able access its data, so I need to have some way to control where these end up running.

As an example, let’s say we have a cluster of 3 EC2 instances managed by Amazon ECS.

  • instance1
  • instance2
  • instance3

Then we have 2 containerized apps:

  • app1, which is stateless
  • app2, which depends on an EBS volume

app1 can run in any instance of the cluster, no problem.

For app2, let’s say we attach and mount the EBS volume in instance2.

Now, the question is: can I put a restriction when defining or launching app2 to be placed only in instance2 so that it can have access to the EBS volume?

Thanks so much in advance!

EDITED:

I’ve reading a bit further and I found a way to do it in Amazon’s documentation. There is a note in the document “Using Data Volumes in Tasks” that says:

Important

Amazon ECS does not sync your data volumes across container instances. Tasks that use persistent data volumes can be placed on any container instance in your cluster that has available capacity. If your tasks require persistent data volumes after stopping and restarting, you should always specify the same container instance at task launch time with the AWS CLI start-task command.

So, looking at the start-task command on the CLI I found out that it is exactly what I was looking for:

start-task

Starts a new task from the specified task definition on the specified container instance or instances. To use the default Amazon ECS scheduler to place your task, use run-task instead.

It has two required parameters that are the task definition you want to launch and the container instance IDs (up to 10) on which the task should run.

So calling the command would look something like this:

aws ecs start-task --task-definition “hello-world:1” --container-instances “1c66549d-b41c-4439-dd43-c2e1c9a2cc2a”

I tried it and it worked perfectly.

Currently, this looks like the only way to do it, since the AWS web UI does not provide a field to specify the container instances when launching a task.

I hope it can be useful somebody.

2
amazon-web-servicesamazon-ec2amazon-ecs
From what I've read: you can't do this. The cluster has a set of instances bound to it that the containers are pinned to based on resources via the scheduler. I think the only way you'd do this is via multiple clusters. - Marc Young
Hi @MarcYoung, I have edited the question with a solution I found. Please take a look a it as there is actually a way to do it without the need to create multiple clusters. - adrianmo

2 Answers

8
votes

ECS now supports Placement Constraints.

See http://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-placement-constraints.html.

So the steps are:

  • Define a custom attribute for each instance (eg , Name=App1) instructions here
  • Define Constraint for each task definition(eg, MemberOf = attribute:Name =~ App1)

It should work.

4
votes

Amazon recently launched a new service, Elastic File System (EFS) which is designed for synchronizing data across many instances, and therefore any container.

The introduction of EFS is an admission that you simple can't use EBS easily for container workloads.

Here's a primer and tutorial about using EFS with ECS.

EFS is built upon NFS. The NFS protocol and the EFS implementation do have some challenges around network IO and data throughput, so it might not be suitable for your workload.