Why is my cluster not honouring CLWLPRTY?

Question

I have the following setup:

• Four MQ Queue managers in a cluster (QM1 (full repo), QM2 (full repo), QM3 (partial repo), QM4 (partial repo))
• All cluster sender/receiver channels have the same rank and priority
• each queue manager has a non-clustered alias queue (ALIAS.TO.CLQ)
• ALIAS.TO.CLQ has a base object of ALIAS.TO.FLOW, which is a clustered alias queue
• ALIAS.TO.FLOW has a base object of L.TO.FLOW, which is a local queue.
• Each of the four queue managers are configured as CLWLUSEQ(ANY).
• The clustered alias queues have a DEFBIND of NOT FIXED. Messages are written by IIB (MQ Output Node), which I believe honours this - although I am seeing the same behaviour using MQ Explorer or RFHUtil also.
• Messages are put by the MQ Output Node without specifying a queue manager name
• Each of the four cluster alias queues has a different CLWLPRTY, but the same CLWLRANK.

The aim of this configuration is to force messages to be received by the same local queue, no matter which queue manager the message is initially put to. If that queue manager is unavailable, the second-highest priority should be used instead, etc.

The issue I am finding is that, for three of the four queue managers the CLWLPRTY is being honoured and the message routed to the highest-priority queue manager. But, on the queue manager with the highest priority itself, the message is being routed to the second-highest priority; instead of using the queue on the same queue manager, which has the highest priority.

If I change which queue manager has the second-highest priority, the message always gets routed there from the one with the highest priority - so it's not just coincidence. If two queue instances share the highest priority, it'll never select the one on the same queue manager as itself; it always seems to prefer to go out to the cluster.

What's going on here, and how can I get it to always honour the priority no matter where the message is put? I have looked at the "cluster workload management algorithm" page in the IBM documentation (https://www.ibm.com/support/knowledgecenter/en/SSFKSJ_9.0.0/com.ibm.mq.ref.con.doc/q082390_.html) but I can't see where I'm going wrong.

--Update--

I have tried creating a new set of queues, in the same format, but only on one of the cluster queue managers - so, ALIAS.TEST -> AL.TEST (cluster but only one instance exists) -> L.TEST. When I put to ALIAS.TEST, I get an error message:

An MQOPEN or MQPUT1 call was issued, specifying an alias queue as the target, but the BaseObjectName in the alias queue definition resolves to a queue that is not a local queue, or local definition of a remote queue. (AMQ4480) An MQOPEN or MQPUT1 call was issued, specifying an alias queue as the target, but the BaseObjectName in the alias queue definition resolves to a queue that is not a local queue, or local definition of a remote queue. (AMQ4480)

Severity: 20 (Error)

Response: Correct the queue definitions.

But I can put to the AL.TEST queue without any issue.

--Edit--

OK, the answer to why it happens appears here: https://www.ibm.com/support/knowledgecenter/en/SSFKSJ_9.0.0/com.ibm.mq.pro.doc/q003120_.html

An alias cannot directly resolve to another alias on the same queue manager.

So how can I get the behaviour I'm wanting?

Answer 1

Based on the following flow where each IIB instance is putting to a local queue manager:

IIB1 -> QM1 -> QLOCAL(L.TO.FLOW) CLWLRANK(0) CLWLPRTY(4) CLWLUSEQ(ANY)
IIB2 -> QM2 -> QLOCAL(L.TO.FLOW) CLWLRANK(0) CLWLPRTY(3) CLWLUSEQ(ANY)
IIB3 -> QM3 -> QLOCAL(L.TO.FLOW) CLWLRANK(0) CLWLPRTY(2) CLWLUSEQ(ANY)
IIB4 -> QM4 -> QLOCAL(L.TO.FLOW) CLWLRANK(0) CLWLPRTY(1) CLWLUSEQ(ANY)

*Assuming the CLWLRANK, CLWLPRTY, and NETPRTY on all CLUSRCVR channels is the same on all four queue managers.

The messages PUT by any of the four IIB instances will always flow to the available queue with the highest CLWLPRTY.

A queue instance can be unavailable from a cluster workload management algorithm perspective for a number of reasons. I have referenced the Knowledge center at the end of this post that provided a detailed explanation of the algorithm. Two common causes would be:

1. The queue manager is down

2. The queue is PUT(DISABLED)

---

Each IIB instance will only GET from the local queue on the same queue manager the IIB instance is connected too. In the setup depicted above this would mean under normal conditions if all four queue managers are up, only the IIB instance connected to QM1 would be receiving messages.

---

What can make a instance of the queue not available is found in the IBM MQ v9.0 Knowledge center page "Reference>Configuration reference>IBM MQ cluster commands>Workload balancing in clusters>The cluster workload management algorithm", I have provided those that could apply to the setup described in your question:

The algorithm steps through the following rules to eliminate destinations from the list of possible destinations.

1. If a queue or topic name is specified:

   a. Queues that are not put enabled are eliminated as possible destinations.

3. When choosing a queue, if the resulting set of queues contains the local instance of the queue, the local instance is typically used. The local instance of the queue is used if one of these three conditions are true:

   • For locally defined queues that are defined with CLWLUSEQ(ANY), or which inherit that same setting from the queue manager, the following points are true, within the wider set of conditions that apply:

     a. The local queue is chosen, based on the status of the locally-defined CLUSRCVR channels in the same cluster as the queue. This status is compared to the status of the CLUSSDR channels that would take the message to remotely defined queues of the same name.

     For example, there is one CLUSRCVR in the same cluster as the queue. That CLUSRCVR has STOPPING status, whereas the other queues of the same name in the cluster have RUNNING or INACTIVE status.

     In this case the remote channels will be chosen, and the local queue is not used.

     b. The local queue is chosen based on the number of CLUSRCVR channels, in any comparison with CLUSSDR channels of the same status, that would take the message to remotely defined queues of the same name.

     For example, there are four CLUSRCVR channels in the same cluster as the queue, and one CLUSSDR channel. All the channels have the same status of either INACTIVE or RUNNING.

     Therefore there are five channels to choose from, and two instances of the queue. Four fifths (80 percent) of the messages go to the local queue.

6. If only remote instances of a queue or topic remain, resumed queue managers are chosen in preference to suspended ones.

7. If more than one remote instance of a queue or topic remains, all channels that are inactive or running are included. The state constants are listed:

   • MQCHS_INACTIVE

   • MQCHS_RUNNING

8. If no remote instance of a queue or topic remains, all channels that are in binding, initializing, starting, or stopping state are included. The state constants are listed:

   • MQCHS_BINDING

   • MQCHS_INITIALIZING

   • MQCHS_STARTING

   • MQCHS_STOPPING

9. If no remote instance of a queue or topic remains, all channels that are being tried again are included. The state constant is listed:

   • MQCHS_RETRYING

10. If no remote instance of a queue or topic remains, all channels in requesting, paused, or stopped state are included. The state constants are listed:

    • MQCHS_REQUESTING

    • MQCHS_PAUSED

    • MQCHS_STOPPED