Azure WebJobs SDK - Create trigger on AWS Queue?

Question

I am building an Azure WebJob (C#) that polls an Amazon SQS queue for new messages and offloads to Azure Event Hub. I have a proof-of-concept that creates a long-polling connection (10s), processes multiple messages and sends to the Event Hub successfully.

I'm trying to determine how to set up the WebJob to run continuously. I am considering creating a TimerTrigger that fires every 10s. However, after looking at the WebJobs SDK, I'd prefer to create a custom event trigger that gets notified when there is a new message in the SQS queue. Is this possible or should I stick to the timer trigger?

I've been successful using default event triggers with Service Bus and Azure Queues but this is my first attempt at creating a trigger from AWS.

Answer 1

I'd prefer to create a custom event trigger that gets notified when there is a new message in the SQS queue. Is this possible or should I stick to the timer trigger?

Yes, but it will have lots of work to do. Before doing it, you could learn how Azure Queue Trigger works by viewing the source code of Azure WebJobs SDK - Queues Listeners.

Azure WebJobs SDK - Queues Listeners

In short, the listener of queue is based on timer and long polling. To simply your work, you could stick to the timer trigger and use long polling when you send request to AWS SQS. By using long polling, it will wait for a period of time until a message is found.

Amazon SQS Long Polling

Answer 2

Writing a custom binding would not only make your usage more natural (once all the work is done creating the custom binding of course), but would also allow others to use your binding :) We have a wiki page Binding Extensions Overview on how to get started writing custom bindings, along with links to sample bindings and starter projects. The general binding pipeline is described in The Binding Process.

You can also look at the source code for any of the other bindings to see how they work, debug through them, etc. Many of the bindings live in the azure-webjobs-sdk-extensions repo, so you can see their inner workings there. Other core bindings live in azure-webjobs-sdk.

I'd personally love to see this Amazon Queue binding get written - you should give it a go! Feel free to connect with us in our extensions repo here with any questions/issues.