Reading Azure Service Bus Queue

1
votes

I'm simply trying to work out how best to retrieve messages as quickly as possible from an Azure Service Bus Queue.

I was shocked that there wasn't some way to properly subscribe to the queue for notifications and that I'm going to have to poll. (unless I'm wrong in which case the documentation is terrible).

I got long polling working, but checking a single message every 60 seconds looks like it'll cost around £900 per month (again, unless I've misunderstood that). And if I add a redundant/second service to poll it'll double.

So I'm wondering what the best/most cost efficient way of doing it is.

Essentially I just want to take a message from the queue, perform an API lookup on some internally held data (perhaps using hybrid services?) and then perhaps post a message back to a different queue with some additional information .

I looked at worker roles(?) -- is that something that could do it?

I should mention that I've been looking at doing this with node.js.

2
node.jsazureazureservicebus
Worker roles have nothing to do with service bus - they're simply stateless Windows Server VM's with specific ways to configure and scale. - David Makogon
How did you get your price ? Service bus queue is available for free. and the cost of a message is nothing - Thomas
Thanks for the responses. I'd seen somewhere I think that worker roles could get notified of queue additions which is what made me ask. For pricing it was under brokered connections for http calls which I'm assuming is what polling is using. Only the first 1000 are free under Standard Tier. azure.microsoft.com/en-gb/pricing/details/service-bus Have I got that wrong as that would be great. Thanks again. - Darren
Seems I may have it wrong.. The FAQ suggests that charges are for "the peak number of concurrent brokered connections" and that "Peaks are measured on an hourly basis, prorated by dividing by 744 hours in a month". They give an example as follows: "10,000 devices receive messages from a Service Bus Queue via HTTP, specifying a non-zero time-out. If all devices connect for 12 hours every day, you will see the following connection charges 10,000 HTTP receive connections * 12 hours per day * 31 days / 744 hours = 5,000 brokered connections." - Darren
So if my understanding is correct, to have a script performing a long poll every 55 seconds (max allowed), that only counts as one "concurrent" connection right? Even if it's polling every 55 seconds? So that's 1 device * 24 hours * 31 days / 744 hours = 1 out of my allowed 1000 right? I suppose the question still stands -- is this the best/only way to do it? - Darren

2 Answers

2
votes

Check out these videos from Scott Hanselman and Mark Simms on Azure Queues. It's C# but you get the idea.

https://channel9.msdn.com/Search?term=azure%20queues%20simms#ch9Search

Touches on:

  • Storage Queues vs. Service Bus Queues
  • Grabbing messages in bulk vs. one by one (chunky vs. chatty)
  • Dealing with poison messages (bad actors)
  • Misc implementation details
  • Much more stuff i can't remember now

As for your compute, you can either do a VM, a Worker Role (Cloud Services), App Service Webjobs, or Azure Functions.

The Webjobs SDK and Azure Functions bot have a way to subscribe to Queue events (notify on message).

(Listed from IaaS to PaaS to FaaS - Azure Functions - if such a thing exists).

Azure Functions already has sample code provided as templates to do all that with Node. Just make a new Function and follow the wizard.

If you need to touch data on-prem you either need to look at integrating with a VNET that has site-to-site connectivity back to your prem, or Hybrid Connections (App Service only!). Azure Functions can't do that yet, but every other compute is a go.

https://azure.microsoft.com/en-us/documentation/articles/web-sites-hybrid-connection-get-started/ (That tutorial is Windows only but you can pull data from any OS. The Hybrid Connection Manager has to live on a Windows box, but then it acts as a reverse proxy to any host on your network).

2
votes

To deal with Azure ServiceBus Queue easily, the best option seems to be Azure Webjob.

There is a ServiceBusTrigger that allows you to get messages from an Azure ServiceBus queue.

For node.js integration, you should have a look at Azure Function. It is built on top of the webjob SDK and have node.js integration :

In the second article, there is an example on how get messages from a queue using Azure Function and nodejs :

module.exports = function(context, myQueueItem) {
    context.log('Node.js ServiceBus queue trigger function processed message', myQueueItem);
    context.done();
};