According to the IBM Knowledge Center page on backout behavior,
If the backout threshold value is zero, poison message handling is
disabled, and poison messages remain on the input queue. Otherwise,
when the backout count reaches the threshold value, the message is
sent to the named backout queue. If the backout count reaches the
threshold value, but the message cannot go to the backout queue, the
message is sent to the dead-letter queue or it is discarded.
The intent is that you would, in fact, be able to manage redelivery in the app code using the JMSX_DELIVERY_COUNT
or JMS_IBM_MQMD_BackoutCount
.
I suspect that the Knowledge Center is at least slightly wrong in that it says the message failing requeue will be discarded. I believe that applies only to non-persistent messages and have submitted feedback asking for clarification.
However, my understanding of the backout behavior is that it is described correctly in the Knowledge Center and that the behavior you are seeing is not what is expected from IBM's JMS classes. There are two possibilities. The first is that the unexpected behavior is implemented in the Spring code rather than IBM's JMS code. The second is that MQ JMS has a bug and isn't behaving as per the docs.
Usually I'd apply the "what is more likely?" test and "MQ bug" doesn't rise to the top of that list. However, I don't know Spring, the likelihood of dedicated poison message handling code, or the general quality of Spring code. So I'd recommend either using stand-alone JMS code to perform the same test, or open a PMR and work with IBM Support to trace the code and isolate the classes where the backout fails.