How do you support FIFO message ordering with Azure Service Bus partitioned queues/topics?

9
votes

I assume that the trade off of using partitioned queues and topics, is that the message ordering is no longer guaranteed.

As the messages are by default sent round-robin to each of the fragments/partitions, then it would mean the message ordering is no longer guaranteed. Can anybody confirm if this is the case?

How can you guarantee message ordering when receiving trades from a partitioned queue.

Is the only way to support FIFO message ordering with partitioned queues/topics, to use sessions? I would assume that all messages for the same session/partition key would at least be delivered FIFO?

3
azureservicebusazureservicebus
Did you ever figure this out? – dreza
stackoverflow.com/questions/7430036/… – Mohit Verma
@MohitVerma that thread talks about azure storage queue, and this is about service bus queue. Totally different. – Nikhil Vartak
See [Azure Service Bus: Handling FIFO using Sessions ](jaliyaudagedara.blogspot.com/2021/04/…) – Michael Freidgeim

3 Answers

8
votes

I found this documented in a blog post.

Hope it helps!

Partitioned Service Bus Queues and Topics

SessionId. If a message has the SessionId property set, then Service Bus uses the SessionId property as the partition key. This way, all messages that belong to the same session are assigned to the same fragment and handled by the same message broker. This allows Service Bus to guarantee message ordering as well as the consistency of session states.

2
votes

Just because you're not using a partitioned queue or topic does not mean you will get FIFO. If you have more than one reader or do async then no, you won't get FIFO UNLESS you use Sessions as was pointed out above. Please use sessions.

1
votes

There is an important point missing about FIFO in the answers above.

When a message comes in to a topic/queue which doesn't have partitioning enabled, then FIFO is observed for message delivery*.

When you enable partitioning on a topic/queue and the SessionId is used for the partitioning key, then messages are no longer guaranteed to be FIFO in relation to each other, they are only guaranteed to be FIFO in relation to the partition they were divvied in to.

Fun fact, partitioning in general can have some interesting side-effects if you have a low number of subscribers to the same subscription/queue, as the partition reader assignments are done round-robin style, and if you have more partitions than subscribers, you can see messages being starved (needs verification from SB team, this is empirical from tests I have done on my own because my messages were being starved).

* As @Dan Rosanova pointed out above, if you have async processing or or multiple readers, then your message processing can't be guaranteed to be FIFO, but the order in which the messages were distributed to the processors is going to be FIFO.

When you use a Session message handler (which requires SessionId to be populated), you're taking it another step further and you're guaranteeing that the messages are processed in order, as the Session message handler takes a lock on the SessionId+MessageId rather than just the MessageId, thus ensuring no other messages within the same session are received by another processor.