How to ensure JMS message redelivery when client crashes

1
votes

I need to ensure redelivery of JMS messages when the consumer fails, but the strategies accepted on similar questions here might not apply to my case.

Consider a JMS client - spring + activeMq - receiving messages, which cannot be lost or duplicated. Because the processing of the messages is expensive, the client batch them. The scenario plays as below:

  • T1 - Client receives messages A, B, C and D
  • T2 - A thread wakes up on the client and decides to handle messages A, B and C
  • T3 - Client receives messages E and F
  • T4 - The thread returns having processed A, B and C. It can then pick up the next batch if another thread has not done so already.

The way the producer is set up now - DefaultJmsListenerContainerFactory and Session.AUTO_ACKNOWLEDGE - messages are removed as soon as they are delivered, whether they have been picked up by a thread or not. If the client goes down just before T4 , messages A to F are lost.

I was planning to use Session.CLIENT_ACKNOWLEDGE on the producer and get each thread to call msg.acknowledge() after T4, but according to the documentation, this will also acknowledge E and F, which will be lost if the client fails after T4.

I am not convinced a transacted session will help here either, as it would cover messages A to F, while a thread completion only guaranties a subset of this has been processed.

My objective is to have a guarantee that in case of a failure of the client, e.g. the VM goes down, all the messages in flight that have not been successfully handled by a thread remains in the topic/queue. They can then be picked up by the client when it comes back up.

Any idea on how I can achieve this ?

S

1
javaspringjmsbatch-processingspring-jms

1 Answers

1
votes

The first thing i want to mention is that you should not use CLIENT_ACKNOWLEDGE cause you will make even worse and increase lose rate. Client acknowledge as u've said also will acknowledge all the messages in the related local jms session otherwise all your fetched messages from one batch.

messages are removed as soon as they are delivered, whether they have been picked up by a thread or not

Usually acknowledge is send back when onMessage method invocation has finished. What thread did u mean? If your processing is done in the same thread then it seems that it will send ack only after it's succeeded with invokation.

How did you achieve batch consuming? If u meant active mq consumer (http://activemq.apache.org/performance-tuning.html) and you are using provided by Spring (WS or Integration) and AMQ code base then you should put your efforts only on proper configuration and not to make some complex custom implementation to achieve 99.999999% success rate. Every jms broker lose messages or have duplicates and youll never get 0 lose rate. And for most enterprise systems it is a normal case when for exampe 1 out of million messages got lost or duplicated.

To get somehow reliable delivery you should use persistent delivery mode and jms transactions anyway. For me duplicated messages are better than completely lost (actually all system architecture should be build idempotent for treating such things). And as you've said and as also mentioned is AMQ docs batch is a tradeoff between reliability and performance. And fundamentally there is no chance to achieve both properties simultaneously. At least with AMQ and jms. So it is after you to decide what is more important for you reliability or performace. The most optimal choice for you will be to define that tradeoff and decide what batch amount of duplicated messages is acceptable in case of lost or duplicate messages