How do I integrate google app engine, taks queue and google compute engine?

Question

I been trying to understand how I can setup the follow architecture on Google's cloud:

• Google app engine receives HTTP request
• Google app engine queues a pull task as a result of the HTTP request
• The task is received by a auto scaling google compute engine instance group

Are there solutions that someone can point me to how to setup a auto scaling task pull task queue handler? Each of my tasks will take approximately a minute to process I estimate.

Answer 1

GCE has a new feature called autoscaler that you can use to automatically scale a group of GCE instances, the documentation is here GCE Autoscaler. It should work well for this use case.

There's also a REST API for pull queues where you can pull from outside of app engine Pull Queue. That will let you pull tasks from the queue using compute engine.