IBM WebSphere MQ request/reply scenario

Question

I'm currently working on a project where I need to interface with an IBM system which communicates with the outside world through WebSphere MQ. I need to query the system in a "request-response" fashion using queues, and I will be doing this through a queue manager.

However, I can't quite get my head around how this works in practical terms.

Say I've got multiple instances of the same application which puts a message onto a request queue. The message gets a CorrelationId and MessageId upon leaving the application, and a ReplyToQueue property gets set on each message to make sure that the queue manager knows which queue to put the response onto.

However, how does the queue manager operate the response queue? There is no guarantee with regards to the timing of responses, so how is it that the correct response gets back to the application instance which issued the corresponding request?

I keep thinking of message queues as a FIFO queue where messages must be picked one by one. However, this would mean that instance A could pick a response meant for instance B. Obviously, this can't be how it works.

Then, when I look at the API (com.ibm.mq.MQQueue) I see that to pick a message I have the chance to provide the CorrelationId and MessageId of the request message. Does this mean that when I query the queue manager for a message (with these ID's set), the queue manager iterates through the messages in the queue and returns the matching message? This, on the other hand, would mean that we're not talking about a FIFO queue?

Answer 1

It is common practice to use CorrelationId to relate request and response messages. It works this way:

1) Let's assume there are two queues, a REQQ - request queue where messages are put asking another application, service application, to pick up and process and a RESPQ to which the service application puts replies.

2) The requester application (let's call it as REQAPP) puts a request message (REQMSG) to request queue (REQQ). Before sending the message the REQAPP sets ReplyToQ property on the messages to RESPQ. When the request message is sent, the JMS provider updates the sent message with a unique MessageId. The requester application caches this unique MessageId for later use.

3) On the other side of the world, the service application retrieves the REQMSG from REQQ, reads the MessageId and ReplyToQ property from that message, processes the request and prepares appropriate response message RESPMSG. It then sets the CorrelationId property of the RESPMSG to the MessageId read from the REQMSG and puts the reply to ReplyToQ.

4) The requester application when reading replies, it uses the caches MessageId and sets selection criteria as below to read reply message

GET MESSAGE FROM RESPQ WHERE RESPMSG.CORRELATIONID==REQMSG.MESSAGEID

This selection criteria makes sure that correct response messages is retrieved. The key here is the service application must set the CorrelationId on the response message to the MessageId of request message.

Although Queue is FIFO type, MQ implementation provides message delivery on a Priority basis where messages with high priority are delivered first or FIFO basis where message on top of the queue is delivered first. Messages can also be retrieved using selection criteria as explained in above example.

Hope this helped

Answer 2

Shashi's answer is spot on for many situations, and is in fact the primary method of distributing responses amongst multiple, relatively low volume requesting applications.

A better choice for high-volume services would involve multiple RESPQs. You can provide a separate RESPQ for each instance of REQApp by using Temporary Dynamic Queues to receive the RESPMsgs -- each instance of REQApp would create the TDQ using the MQOPEN function, and specify the RESPQ name in the ReplyToQ attribute of every message it sends out.

With this setup you would not have to worry about sequencing and correlation IDs, as the messages will be returned to the individual RespQs in the same order in which they are processed by the service application.

Of important note here is that TDQs are just that - temporary. When the OPENing application closes the queue - either via MQCLOSE or by terminating - any messages in the TDQ are lost and not recoverable. If this is a issue - that all responses must be processed no matter what - then you would want to one-time allocate a series of permanent dynamic queues (also using MQOPEN), one for each instance of REQApp, and each instance of REQApp must reconnect to that same queue each time.

I hope this helps as well.