I believe task queues (push, pull, deferred) in Google App Engine do not guarantee that tasks will be executed in FIFO order. For example, suppose I have a task queue with tasks A, B, and C, and each task has timestamps t_A, t_B, and t_C, such that t_A < t_B < t_C. How can I ensure that tasks A, B, and C are executed in order of timestamp? If task B fails, I would like to delay the execution of task C until task B executes successfully. I've seen an ETA field to set the earliest time that a task can be sent, but this seems like more of a heuristic, not a guarantee.
2
votes
considered calling the next task from the previous?
- Paul Collingwood
If I understand correctly, this would basically mean that I would be managing task execution and the workflow myself. I was hoping I could stick with Google App Engine Task Queues. But Google App Engine Task Queues and Tasks do not provide an 'execute()' method in the API, which seems to forbid one from executing a task in the queue from another task.
- morfys
rather you can create a task as the last thing that the current task does.
- Paul Collingwood
1 Answers
2
votes
Consider using the Pipelines API:
https://github.com/GoogleCloudPlatform/appengine-pipelines
They have already done the work for you:
class LogWaitLogInOrder(pipeline.Pipeline):
def run(self, message1, message2, delay):
with pipeline.InOrder():
yield LogMessage(message1)
yield Delay(seconds=delay)
yield LogMessage(message2)
yield LogMessage('This would happen immediately on run')
The pipelines api also gives you reporting/feedback/notification/etc.
But if that's overkill for your particular needs, then just do as @Paul suggested, just create the next task when the first one completes and hope for the best
video reference:
Google I/O 2010 - Data pipelines with Google App Engine:
