Multiple Azure WebJob functions on single queue message?

Question

Our back-end is collecting raw data and pushing it to Azure Storage Queue. We would like to do two things with each queued message:

Log/archive it
Parse it and send the parsed result to a new queue

To keeps things small and clear, we would like to have two WebJob functions pointing to the same queue:

    public static void ArchiveRawData([QueueTrigger("raw")] RawData data, [Blob("{Ticks}.dat")] out string raw)
    {
        raw = data.Data;
    }

    public static void ParseRawData([QueueTrigger("raw")] RawData data, [Queue("result")] out Parsed parsedData
    {
        var parsed = Parser.Parse(data.Data);
        parsedData = parsed;
    }

But this doesn't work: Either the ArchiveRawData or ParseRawData gets the message, but not the other.

Is there an option somewhere which would make the above scenario work? It seems that the message is now automatically dequeued after the first function finishes (no matter which one). But I think the WebJobs SDK could detect that there are multiple functions with same QueueTrigger and it could dequeue the message only after all the functions have completed.

In order to get around this, we currently have two outputs in a single function:

    public static void ParseRawData([QueueTrigger("raw")] RawData data, [Queue("result")] out Parsed parsedData, [Blob("{Ticks}.dat")] out string raw)
    {
        var parsed = Parser.Parse(data.Data);
        parsedData = parsed;
        raw = data.Data;
    }

But as I said, we would like to keep things small and simple so it would be great if we could use separate function.

Victor Hurdugaci Victor Hurdugaci · Accepted Answer · 2015-04-13T16:38:55

Unfortunately, the SDK does not support multiple functions listening to the same queue.

If you want multiple functions to be invoked, just create a few methods and make the webjob function the entry point. Then the webjob will call those functions.

Alternatively, you can have the second function listen to a different queue. The first queue, adds a message to this queue.

Multiple Azure WebJob functions on single queue message?

2 Answers