Azure communication between Worker Role and Web Role

Question

Hi I am building a Cloud Service which has (for the moment) one web and one worker role. My desired workflow would be: the browser calls a webApi controller on the web role, which sends a message to a queue (or service bus) which is then processed by the worker role. So far so good. Now when the worker role finishes processing the message I would like to call a method on the web role, who will then signal the browser that the processing has completed (via SignalR). Please excuse me if this is not the right place to ask since this is more like a “best practice” question rather that a real problem. I have so far considered 2 approaches:

1. The worker role updates a table row (in the table storage) with the progress and the completion of the task. There is no signaling to the web role. The browser reads polls the table storage directly (via REST api) and therefore knows when the task has completed. This works well (I have already tested it), even though I don’t like the approach of constant polling and I would like to have an “event-based” solution. Moreover, once the client gets the info that the process has finished, it must perform an additional call to a web api method to broadcast to the other clients (via SignalR) that the operation was completed.

2. Using Interrole communication together with SignalR (see code sample below) also works (already tested as well)

Code sample:

var protocol = "http";      
var ipAddress = RoleEnvironment.Roles["XXX.YYY.Web"]
        .Instances[0]
        .InstanceEndpoints.ToArray()
        .Where(ep => ep.Value.Protocol == protocol)
        .First()
        .Value.IPEndpoint.ToString();

var stringEndpoint = string.Format("{0}://{1}", protocol, ipAddress.ToString());                
Trace.WriteLine("Retrieved endpoint address: " + stringEndpoint, "Information");            
HubConnection connection = new HubConnection(stringEndpoint);
IHubProxy proxy = connection.CreateHubProxy("MyWebHub");
connection.Start().Wait();

//later...

proxy.Invoke("ProgressUpdated", clientId, progress);

My question is: are there other (better) ways to communicate in direction Worker role -> Web role? That is, trigger a method on the web role when a worker role has finished its processing? The method on the web role would then broadcast the update to all clients via SignalR. I have also taken a look at Event Hubs but to my understanding the Event Consumers would still run on the worker role.

Answer 1

Ok after some extra attempts and a bit of research I have come up with a possible solution... I was using this code

AzureServiceBus.QueueClient.OnMessage((message) =>
            {
                try
                {
                    // Process message from queue
                    Trace.WriteLine("Message received: " + ((BrokeredMessage)message).Label);

                    var breezeController = new BreezeController();
                    breezeController.TestSignal();

                    // Remove message from queue
                    message.Complete();
                }
                catch (Exception)
                {
                    // Indicates a problem, unlock message in queue
                    message.Abandon();
                }
            });

inside the OnStart method of the Web Role (NOT the Worker role) so that I could reference my method inside the web role (TestSignal() in this case) but it turned out that the IHubContext was always null when called from within this OnMessage event handler, since it belongs (very likely) to a different AppDomain thus even the static Hub of signalR was not shared. I have therefore moved this same whole code inside the Global.asax.cs so that it will share the same AppDomain and now it works. I think I will stay with this approach since I like it much more than the continuous polling.

Answer 2

I think you already have enough knowledge and implementation experience to achieve it. Using SignalR is brilliant approach, I learned from you.

Another little-bit-different approach, I use constant polling by using Azure Scheduler in order to send GET message to WebRole.

IMO, as a nature of webapi server, polling is the most suitable and reliable approach to follow what web server is designed.

Answer 3

A little late to the question :) but what we have implemented is to specify a response queue and session ids to which the web role awaits after sending the message to the worker role. You can tweak it to avoid blocking the response while the web role awaits the worker role reply (in our scenario we wanted to specifically wait)

WebRole

string sessionId = Guid.NewGuid().ToString(); 
[...]
// put message in sync queue
var message = new BrokeredMessage(request)
{
   ReplyToSessionId = sessionId
};
await ServiceBusConnector.Instance.SyncClient.SendAsync(message);

// now listen to reply on session response queue (only accepts message on same session id)
MessageSession session = await ServiceBusConnector.Instance.SyncResponseClient.AcceptMessageSessionAsync(sessionId);

BrokeredMessage responseMessage = await session.ReceiveAsync(TimeSpan.FromMinutes(5));
await responseMessage.CompleteAsync();
await session.CloseAsync();

Response response = responseMessage.GetBody<Response>();
// process Worker Role's response

Worker Role

    // if a ReplyToSessionId has been specified, it means that the sender is
    // actively waiting for a response
    if (!String.IsNullOrEmpty(receivedMessage.ReplyToSessionId))
    {
        // now respond on sync response queue
        var responseMessage = new BrokeredMessage(response)
        {
            SessionId = receivedMessage.ReplyToSessionId
        };

        // consider allowing client to specify a ReplyTo response queue (not needed for now)
        await ServiceBusConnector.Instance.SyncResponseClient.SendAsync(responseMessage);
    }

Answer 4

Also check out Jessie's approach here to directly communicate back to the app via HttpClient.

public class Functions
{
    public static async Task ProcessQueueMessage([QueueTrigger("jobqueue")] Guid jobId, TextWriter log)
    {
        for (int i = 10; i <= 100; i+=10)
        {
            Thread.Sleep(400);

            await CommunicateProgress(jobId, i);
        }
    }

    private static async Task CommunicateProgress(Guid jobId, int percentage)
    {
        var httpClient = new HttpClient();

        var queryString = String.Format("?jobId={0}&progress={1}", jobId, percentage);
        var request = ConfigurationManager.AppSettings["ProgressNotificationEndpoint"] + queryString;

        await httpClient.GetAsync(request);
    }
}