NServiceBus and remote input queues with an MSMQ cluster

Question

We would like to use the Publish / Subscribe abilities of NServiceBus with an MSMQ cluster. Let me explain in detail:

• We have an SQL Server cluster that also hosts the MSMQ cluster. Besides SQL Server and MSMQ we cannot host any other application on this cluster. This means our subscriber is not allowed to run on the clsuter.
• We have multiple application servers hosting different types of applications (going from ASP.NET MVC to SharePoint Server 2010). The goal is to do a pub/sub between all these applications.
• All messages going through the pub/sub are critical and have an important value to the business. That's why we don't want local queues on the application server, but we want to use MSMQ on the cluster (in case we lose one of the application servers, we don't risk losing the messages since they are safe on the cluster).

Now I was assuming I could simply do the following at the subscriber side:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <configSections>
    <section name="MsmqTransportConfig" type="NServiceBus.Config.MsmqTransportConfig, NServiceBus.Core" />
    ....
  </configSections>

  <MsmqTransportConfig InputQueue="myqueue@server" ErrorQueue="myerrorqueue" 
        NumberOfWorkerThreads="1" MaxRetries="5" />
  ....
</configuration>

I'm assuming this used to be supported seen the documentation: http://docs.particular.net/nservicebus/messaging/publish-subscribe/

But this actually throws an exception:

Exception when starting endpoint, error has been logged. Reason: 'InputQueue' entry in 'MsmqTransportConfig' section is obsolete. By default the queue name is taken from the class namespace where the configuration is declared. To override it, use .DefineEndpointName() with either a string parameter as queue name or Func parameter that returns queue name. In this instance, 'myqueue@server' is defined as queue name.

Now, the exception clearly states I should use the DefineEndpointName method:

        Configure.With()
                 .DefaultBuilder()
                 .DefineEndpointName("myqueue@server")

But this throws an other exception which is documented (input queues should be on the same machine):

Exception when starting endpoint, error has been logged. Reason: Input queue must be on the same machine as this process.

How can I make sure that my messages are safe if I can't use MSMQ on my cluster?

Dispatcher!

Now I've also been looking into the dispatcher for a bit and this doesn't seem to solve my issue either. I'm assuming also the dispatcher wouldn't be able to get messages from a remote input queue? And besides that, if the dispatcher dispatches messages to the workers, and the workers go down, my messages are lost (even though they were not processed)?

Questions?

To summarize, these are the things I'm wondering with my scenario in NServiceBus:

1. I want my messages to be safe on the MSMQ cluster and use a remote input queue. Is this something is should or shouldn't do? Is it possible with NServiceBus?
2. Should I use a dispatcher in this case? Can it read from a remote input queue? (I cannot run the dispatcher on the cluster)
3. What if the dispatcher dispatchers messages to the workers and one of the workers goes down? Do I lose the message(s) that were being processed?

Answer 1

Phill's comment is correct.

The thing is that you would get the type of fault tolerance you require practically by default if you set up a virtualized environment. In that case, the C drive backing the local queue of your processes is actually sitting on the VM image on your SAN.

Answer 2

You will need a MessageEndpointMappings section that you will use to point to the Publisher's input queue. This queue is used by your Subscriber to drop off subscription messages. This will need to be QueueName@ClusterServerName. Be sure to use the cluster name and not a node name. The Subscriber's input queue will be used to receive messages from the Publisher and that will be local, so you don't need the @servername.

There are 2 levels of failure, one is that the transport is down(say MSMQ) and the other is that the endpoint is down(Windows Service). In the event that the endpoint is down, the transport will handle persisting the messages to disk. A redundant network storage device may be in order.

In the event that the transport is down, assuming it is MSMQ the messages will backup on the Publisher side of things. Therefore you have to account for the size and number of messages to calculate how long you want to messages to backup for. Since the Publisher is clustered, you can be assured that the messages will arrive eventually assuming you planned your disk appropriately.