Processing Service Bus Queue Messages One-By-One

Question

I've relatively new to Azure Service Bus Queues and am building a project that needs to process messages in a queue in the order in which they arrive (FIFO).

Using Microsoft's documentation, I was able to figure that part out. From what I understand I need to turn sessions on for the queue?

What I'm struggling with is determining what would be the best approach / service stack to perform the following set of ordered tasks against the queue.

First let's assume that we have a FIFO-based Service Bus Queue in place that has n number of messages. How might I:

1. Pick up the first message from the queue.
2. Process the message using an Azure Function.
3. Send a payload to IoT Hub that will deliver to an external device (C2D).
4. This is the part that I can't figure out...Wait for either a Completed indicator to return from IoT Hub or wait until the TTL expires for the outgoing IoT Hub Message.
5. Now complete the item in the queue.
6. Start back at 1.

I would imagine that perhaps a Logic App might help me achieve what I'm attempting to do. Everything seems straight forward up until #4. I can't figure out how to have the logic app 1. Wait until IoT Hub acknowledges that the Cloud-to-Device message was sent or expired and 2. Don't process the next message in the queue until the IoT Hub acknowledgment has been received and I've marked the current queue message as complete.

Please be advised: The reason why I'm so specific about this is that the devices receiving the outgoing IoT Hub C2D messages care about order. If they receive messages out of order, it throws the process off.

Any suggestions is greatly appreciated.

Answer 1

Azure Service bus Queues with Session can be used for achieving ordered processing of the message.

From the question, I can understand that the message from the Service Bus Queue should be removed only after the acknowledgement is received from IOT Hub.

I can sense a problem that may happen with this flow.

First of all, you need to understand about lock duration property in Service Bus Queue. Based on the value set to this property, the messages will be locked for x minutes or seconds. Whenever a message is received from a Queue, a lock will be applied for the message. So that the message will not be available for any other receivers for x minutes or seconds. Complete operation should happen before the lock gets expired. Once the lock is expired, the message will be available for other receivers to process. In your case, there is a chance for the same Azure Function to receive the message once again (resulting in duplication)

The maximum value that can be set for the lock duration property is 5 minutes.

So, if you are sure that the IOT Hub responds with in 5 minutes, you can proceed with this implementation. Still, there will be no option to let the Function know when to process the next message.