Poll-and-sleep on Azure storage queue from WebJob

Question

I know that there is the QueueTrigger stuff that gets invoked when a new message is available in the queue. However, this "Framework style" WebJobs make it complicated for me to initialize my environment correctly.

This is why I want to poll the queue myself, the basic pattern is like this:

var account = CloudStorageAccount.Parse("connectionString");
var client = account.CreateCloudQueueClient();

// Per application initialization stuff goes here

while(true) {
    var queue = client.GetQueueReference("myqueue");
    var message = queue.GetMessage();

    if (message != null) {
        // Per message initialization stuff goes here
        // Handle message

        queue.DeleteMessage(message);
    }
    else {
        Thread.Sleep(10000); // Or some exponential back-off algorithm
    }
}

My question is, are there potential problems with this approach when running continuously in a WebJob? If so, what are possible alternatives to avoid "framework-style" WebJobs?

Edit

As requested by Victor, here is some more information about why I don't want to use the QueueTrigger approach. There are basically two reasons.

The first is already mentioned above, it's the fact that the framework calls a method with the QueueTrigger attribute. This means that I have to put all initialization in that method.

We have a two-stage IoC container (per-application and per-request) in our web apps, and I want to keep the WebJobs as close to the web apps as possible, so I'd like to use that two-stage IoC in the WebJobs, too (the per-application IoC does some heavy initialization that is then reused across all requests). The consequence of this would be that I have to put the per-application container in a static variable or singleton, so that I can access it from the static QueueTrigger method. This is a design quirk I'm not willing to make (IMO, Microsoft does this much too much, e.g. Thread.CurrentCulture or HttpContext.Current - that's REALLY an anti-pattern and hurts testability).

The perfect WebJobs SDK for the situation described above would make the infrastructure (back-off timer, poison-handling, etc.) available as service, so that the main control flow always remains with the application. This may or may not be possible for all features of the SDK, I don't know the SDK well enough to judge that.

The second reason is developer convenience. We have a distributed team that sometimes works in offline environments. From what I've read here and here, there is no possibility to run the WebJob application locally with the storage emulator and have it dequeue messages. With the self-polling approach that I'm taking now, this works like a charm.

Just a note, that if you're concerned about IoC you can use the IJobActivator interface of the WebJobs SDK and wrap your container of choice to do all the instantiation you'd otherwise code up in your webjob function. While I agree that it's a pain we cannot run webjobs against the local storage emulate, we've overcome this by writing our webjobs as console apps. Running them locally and testing them is easy peasy. — Øyvind
@Øyvind small update, there are beta/experimental packages available. More information here — Rik van den Berg

Victor Hurdugaci Victor Hurdugaci · Accepted Answer · 2015-04-11T16:29:55

There is nothing wrong with your approach. It will work and it is very similar to what the WebJobs SDK does under the hood (we have an exponential back-off algorithm).

I am curious, what environment initialization is complicated with the SDK?

Poll-and-sleep on Azure storage queue from WebJob

1 Answers