Basic orchestration with Azure Service Bus?

3
votes

I'd like to understand how much of the following scenario is supported by the Azure Service Bus:

Worker server notifies an unknown number of webservers of an event Each web server processes the message (processing takes sometime - 10-30 minutes) When every web server is done with processing of the first message, all of the web servers need to receive a new event. Basically I'm trying to synchronize a number of web roles after performing a long-running job on each web role.

How much of this can I get "for free" from Azure service bus?

4
azureazureservicebus
Topics sound like it fits the bill, was there something about topics that you think wouldn't fit here? – Peter Ritchie
I'm not an expert in ServiceBus, but trying to find out how much it can help with. Topics sound like the place to start. But can it help with synchronizing or do I need to get my worker role involved? – Igorek
Topics let you implement pub/sub, so one process can notify many other processes (zero or more). They can, in turn, notify all other processes that they've processed the event (although that gets a bit chatty since you're likely notifying one process) reply-to might come in handy here. When 1 process is done processing the event, it can send a message (send to a non-topic queue) to the process that published the event. – Peter Ritchie

4 Answers

1
votes

Based on how i interpret your question, it Sounds to me like the complicated bit would be not sending the 2nd message until you are sure all of the unknown number of servers had processed their message.

While service bus would offer pub/sub and correlation features which could help you here, to me your describing a pattern where servers would register their interest and also acknowledge that they have processed their message so something would then count up the ack's and once everyone had ack'd the 2nd message would be sent.

This orchestration isnt something service bus can help you with. This puts you in the place of building something in a worker role or possibly using BizTalk when its available in a VM role.

Something like a long running BizTalk orchestration to handle the registration, tracking of acknowledgements and publishing a new message combined with a service bus topic for pub sub to the web servers could be a way to do this

3
votes

Azure Service Bus has a lot of rich messaging features to help with both the Pub/Sub aspect of your requirement as well as the request/response correlation. The concept of sessions (grouped/related) messages along with session state can be very helpful here. Following are some specific links that may help: MSDN article on sessions: http://msdn.microsoft.com/en-us/magazine/jj863132.aspx

Sample for using sessions: http://code.msdn.microsoft.com/windowsazure/Brokered-Messaging-Session-41c43fb4

Request/Response sample: http://code.msdn.microsoft.com/windowsazure/Brokered-Messaging-Request-0ce8fcaf

Talk on correlation etc.: http://channel9.msdn.com/Blogs/Subscribe/Service-Bus-Messaging-Deep-Dive

2
votes

You may also want to take a look at the recently released Service Bus Durable Task Framework preview here. For some samples of how to use this framework, see here.

Basically, using this framework on top of Service Bus features such as sessions etc you could write some C# orchestration code that does something roughly like this:

            ...
            // phase 1
            List<Task> taskList = new List<Task>();
            foreach (var serverName in serverList)
            {
                taskList.Add(context.ScheduleTask<object>(typeof(ExecutePhase1OnWebServerActivity), serverName));
            }

            // wait for all of the executions to finish
            await Task.WhenAll(taskList);

            // phase 2
            taskList = new List<Task>();
            foreach (var serverName in serverList)
            {
                taskList.Add(context.ScheduleTask<object>(typeof(ExecutePhase2OnWebServerActivity), serverName));
            }

            await Task.WhenAll(taskList);
            ...
0
votes

This looks to me like a combination of Workflow Foundation and Service Bus.