How to tell when the last message from an Azure Queue is removed

0
votes

I'm exploring azure functions and queue triggers to implement a recursive solver.

The desired behavior is as follows:

  1. There exists a queue called "solveme" which when it receives an item containing state, will create a second queue with a unique name (GUID) and send the initial state to that queue with some correlation information.

  2. When a message is received into the newly created queue, it will possibly enqueue more work representing all the next possible states so that before the message is "completed", the new work will exist in the queue. If there are no valid next states for the given state, it will just complete the message. This means that the original message is not "completed" until all new work is enqueued. During this step, I'm writing off data to azure tables representing the valid solutions using the correlation information from step 1.

3. When all messages have been processed from the newly created queue, delete the queue.

I am not sure how to go about #3 without having a timer or something that checks the queue length until it is zero and deletes it. I'm trying to only use Azure Functions and Azure Storage - nothing else.

It would be nice if there was someway to do directly from a QueueTrigger or within a function called by a QueueTrigger.

Further, is there a better / standard way to trigger completion of this kind of work? Effectively, the empty queue is the trigger that all work is done, but if there is a better way, I'd like to know about it!

1
azurerecursionazure-functionsazure-queues

1 Answers

3
votes

There's no reliable way to do that out of the box. If your consumer is faster than producer, the queue could stay on 0 for long time, while producer still keeps sending data.

I would suggest you to send a special "End of Sequence" message, so when consumer receives this message it knows to go and delete the queue.

P.S. Have a look at Durable Functions - it looks like you could use some of the orchestration there instead of implementing workflows yourself (just mind it's in early preview).