EC2 Instance - Start, Run Process, Stop

Question

I have an application installed on a GPU compute EC2 node that does a lot of heavy calculation. I would like to set it up to do the following:

1. Send request to GPU EC2 node to run a process.
2. If GPU EC2 node is already started, and running a process, wait until that process is finished and run the process from this new request.
3. Else Start the GPU EC2 node and run process from new request.
4. Once all processing has finished, Stop EC2 node.

I've gotten this working partially by starting/stopping instances through Lambda. I am curious, is there a more automatic, built-in, functionality in AWS to accomplish this?

Answer 1

A better architecture might be:

• When there is work to be done, push it to an Amazon SQS queue (or store details in Amazon S3 and push a message that points to Amazon S3)
• Create a CloudWatch Alarm that triggers when the size of the queue is greater than zero
• The alarm can trigger a Lambda function that starts the GPU node
• The GPU instance has software running that pulls work from the SQS queue
• When there is no work remaining, the GPU instance calls shutdown on the operating system