Queue implementation for WCF Service

Question

I need a suggestion on queue design for my WCF service. So, i have a wcf service and a lot of clients. Clients generate messages and puts them into queue for my service to process. Service process them async and clients do not need callback or response on putting message to queue.

My service needs to dequeue message from queue for processing with multi threads. But if my service will stop or failed, or server shutdown immediate - this service has to reprocess message, that did not conitues processing.

So, i consider msmq queue implementation. Msmq lets me make good queue, but how to implement this requirement: reprocess message, that was not processed to the end?

May be somebody wiil give me another betted suggestion. Thanks for any help.

If the service stops, the messages should remain in the queue and be processed when it starts up again, is this not happening? If so, how are you hosting the WCF process that reads MSMQ? — Adam Fyles
But is it possible for service to take message from queue, and start processing. At the same time another processing thread takes another mesage from queue for processing and start process. Then we kill service process and after restart it can found that 2 items in queue not processed? — cyssima
Each thread should be involved in a transaction, so both messages should be there. An alternative is to create multiple processes that point at the same queue instead of having many threads. This is the "Competing Consumer" pattern and is possible using MSMQ 4. — Adam Fyles

Sixto Saez Sixto Saez · Accepted Answer · 2011-04-13T13:00:47

The key here is which server OS the machine hosting your service is running. If you are using Server 2008 (or Vista technically) then you can take advantage of the MSMQ 4 automatic poison message handling. This automatically removes a message from the queue to a special poison queue so the service can continue processing messages. In MSMQ 3, you had to do this in your code. This article on MSDN should give a good start and this article shows the poison message handling code.

I don't see that you need to worry about multi-threading you service code. If you want high throughput then use clustered MSMQ servers and load balanced WCF services, its way more reliable than attempting to roll your own MSMQ based multi-threaded code.

Queue implementation for WCF Service

2 Answers