Azure Service Bus - Delete Specific Message

Question

I'm developing a system which will involve a lot of data synchronisation broken down into small tasks. I am adding each small task as a job/message on the Azure Service Bus queue.

I have X number of Worker Roles then checking the queues and processing the data.

I don't expect to have many messages in the queue because the aim is to process a message, complete it and then re-add the same message again, but scheduled for X minutes time. This will give me a loop to continue to process those tasks.

The great thing about the Azure functionality is that they handle all of the server side stuff for you, but the downside is that it can sometimes be difficult to debug or manipulate the data.

What I want to be able to do is present a list of messages in the queue (Which I have done using PeekBatch) in a web interface. I then want to be able to select some/all of the messages and delete them.

I might want to do this if there is a bug in the code and I want to stop messages of a certain type to stop.

Following on from that I'll have functionality to re-add messages from the web page too. Perhaps I might want to up my worker roles and messages to perform a task at a faster rate (or slow them down), or re-add messages I have deleted.

So, the question is, how can I actually select a specific message from the queue and then delete it? From what I can see, there is no obvious way to do this and, if it's possible, it will require some kind of workaround. This sounds a bit bizarre to me.

Edit:

I've got something that works, but it really doesn't seem a great solution:

    public void DeleteMessages(List<long> messageIds)
    {
        foreach (var msg in Client.ReceiveBatch(100))
        {
            if (messageIds.Contains(msg.SequenceNumber))
                msg.Complete(); // Deletes the message
            else
                msg.Abandon(); // Puts it back in the queue
        }
    }

This will get less and less efficient the larger the queue, but it at least does stop all activities whilst the delete call is going on and deletes the specified messages.

It will also only delete messages which are ready to process. Messages in the future will be ignored, so I've currently added the ability to add "Sleep" messages, to stop processing the queue until my messages are "ready" and I can delete them.

I've been informed by Microsoft that they are currently working on the API to delete specific messages which should be available in a few months. Until then, it's all about workarounds.

June Update:

Still no update from Microsoft on this issue and the above method was less than ideal. I have now modified my code so that:

The object I put into the message has a new property:

Guid? MessageId { get; set; }

Note it is nullable Guid just to be backwards-compatible

When I want to delete a message, I add my MessageId into a database table "DeletedMessage".

When it comes to processing a message, I look in the DeletedMessage table for a matching Guid and if it finds one I simply Complete() the message without doing the normal processing.

This works well, but is a slight overhead. If you are not dealing with huge numbers of messages it's not a big problem.

Also note that I did this originally by using the SequenceNumber, but (bizarrely) the SequenceNumber changes between peeking and retrieving the message! This prevents the idea from working, unless you use your own Id as above.

skoz skoz · Accepted Answer · 2014-11-20T22:17:54

You can create your own queue cleaner program that has a message listener that peeks and either abandons or commits the the message depending on some criteria of the message. It could be an action in a controller if you foresee the need for cleansing on actual production environments as well.

Azure Service Bus - Delete Specific Message

2 Answers