nservicebus + webhooks +Errors +MaxRetries

Question

Feature Description

The NServiceBus gateway, http://docs.particular.net/nservicebus/gateway/, seems to be a way to achieve an internal webhook using the NServiceBus infrastructure.

We need to go further with this concept to open up a few event to any 3rd party subscriber that has access to register a webhook url in our system.

Review

We plan to create two initial window services

1) WebHookBatchService, that can be added as a subscriber to specific messages of interest.

<UnicastBusConfig>
    <MessageEndpointMappings>
           .......
        <add Messages="MyMessages.MyImportantMessage, MyMessages" Endpoint="WebHookBatchService.Queue"/>
           .......
    </MessageEndpointMappings>
</UnicastBusConfig>

2) WebHookProcessService - actually processes 1 message sent by the WebHookBatchService.

Once messages are received on the WebHookBatchService.Queue our WebHookBatchService will look up all the subscribers for the specific tenant + message type and foreach send individual messages to WebHookProcessService.Queue for the WebHookProcessService (which we can make an instance of nservicebus loadbalancer to bridge the batch and actual processor) to actually process the real messages probably using http://restsharp.org/.

Questions

Are there any existing open source projects that do this today?

Now since we have no control of the durability of the subscribers how should we manage errors?

http://wiki.shopify.com/WebHook

A webhook will be deleted if there are 19 consecutive failures for the exact same webhook.

It doesn't mention any delays in the webhook.. What have people experienced with standard delay in retry logic?

Here are some other thoughts:

proposal 0: MaxRetries="1". Purge WebHookProcessService.ErrorQueue nightly. (no retry - guaranteed message loss if it fails the first time)

proposal 1: MaxRetries="1" on exception catch send email containing xml version of the message that would have been delivered over http.
Purge WebHookProcessService.ErrorQueue nightly. -- I see potential a spam issues.

proposal 2: The nservicebus MaxRetries retries right away without delay. So i would need to create (1hr - 24hr) bucket queues and use a RetrySchedulerService although I see this as difficult to maintain and confusing for subscribers when they all at once get 25 messages in a non DateCreated ordered fashion when there service endpoint begins to work.

Digging for ideas...

Answer 1

The Gateway is typically used for communication between physical sites over HTTP. Since you are exposing an endpoint to the world to accept callbacks, I'm thinking you could just use the built-in WCF hosting and expose your endpoint through the firewall to 3rd parties. The rest of your setup sounds appropriate to me.

As for errors, you are correct, NSB retries immediately, but if you using web call backs this may get you by in the cases there are small hiccups. You will need to determine how you want to process the error queues, we just build in a new endpoint to process the error queues with logic to determine the retries, delay etc. A nice way to accomplish this is to use a Saga, which includes a Timeout manager. This enables a workflow where you can retry a specified number of times, try another communication, log everything, and ultimately notify someone who can contact the 3rd party to let them know there stuff is busted.