PHP AWS Elastic Beanstalk background workers

Question

I have deployed my application using Elastic Beanstalk, since this gives me a very easy deployment flow, to multiple instances at once using the "git aws.push".

I like to add background processing support to my application. The background worker will use the same codebase, and simply start up a long lived php script that continuously looks for tasks to execute. What AWS should i use to create such a worker instance?

Should i use the EB for this aswell or should i try to setup a standard EC2 instance (since i dont need it to be public available) ? I guess thats the right way of doing it and then create a deployment flow that make it easy to deploy to both my EC2 worker instances and to Elastic beanstalk app? or is there a better way of doing this?

Answer 1

If the worker is just polling a queue for jobs and does not require an ELB, then all you need to do is work with EC2, SQS, and probably S3. You can start EC2 instances as part of an Auto-scaling group that, for example, is configured to scale as a function of depth of the SQS queue. When there is no work to do you can have minimum # of EC2, but if the queue gets deep, auto-scaling will spin up more.

Answer 2

AWS EB now adds support for Worker Instances. They're just a different kind of environment which those two differences:

• They don't have a cnamePrefix (whatever.elasticbeanstalk.com)
• Instead, they've got a SQS queue bound

On each instance, they run a daemon called sqsd which basically poll their environments' sqs queue and forward it to the local http server.

I believe it is worth a try.