Use MSMQ queue to decouple message producer and consumer?

Question

I would be grateful for some design suggestions concernig a windows service (c#) for publishing reports to a SOAP service.

It fetches a limited set of reports from a database (reports in Oracle AQ table), aggregates them into a message and forwards this message to a WCF SOAP service. Reports are marked as "sent" if they have been transmitted via SOAP successfully. Otherwise they are added again to the AQ table (via a db job).

So I came up with following designs. What would be the best way to go? Would the queue improve the design in terms of scalability, robustness, decoupling? Is it a good idea to use queuing in this case?

Proposed design A: Service with 1 to N threads. Each thread processes reports synchronously (fetching reports, aggregating, translating, sending via SOAP)

Proposed design B: Windows service with:

1 MSMQ message queue
1 to N Producer threads: (fetching reports, aggregating, enqueuing message via MSMQ)
1 to N Consumer threads: (dequeuing, translating, dipatching via SOAP)

Proposed design C:

Windows service with producer threads (fetching reports, aggregating, enqueuing messages to a private MSMQ queue via WCF NetMsmqBinding Client)
IIS/WAS hosted MSMQ-enabled service (listens to the MSMQ queue, dequeuing, translating, dipatching via SOAP)

Jim Mischel Jim Mischel · Accepted Answer · 2013-12-18T22:53:08

Is there a particular reason you chose MSMQ? If you use your proposed design B, you could use BlockingCollection.

I don't see that MSMQ provides a particular advantage in this scenario unless you want multiple processes or you're expecting to spread this out across multiple computers.

But do you really even need multiple threads? It seems like your limiting factor here will be either database access time or communication to the WCF service. Unless the WCF service has to do some major processing before you can call the job successful.

So are you sure you can't just have:

while there are unsent jobs in the database
    get job
    send job to WCF
    if job sent successfully
        mark job as sent
end while

Obviously my knowledge of your situation is limited to what you've posted in your question, so it's possible I've missed something important.

Use MSMQ queue to decouple message producer and consumer?

1 Answers