Rabbitmq: Unacked message not going away after broker restart

3
votes

We have observed the following behavior of RabbitMQ and are trying to understand if it is correct and how to resolve it.

Scenario:

  1. A (persistent) message is delivered into a durable queue
  2. The (single) Consumer (Spring-AMQP) takes the message and starts processing => Message goes from READY to UNACK
  3. Now the broker is shut down => Client correctly reports "Channel shutdown"
  4. The consumer finishes the processing, but can not acknowledge the message as the broker is still down
  5. Broker is started again => Client reconnects

As a result, one message remains unack'ed forever (or until the client is restarted).

Side note: In the Rabbit Admin UI, I can see that two channels are existing now. The "dead" one that was created before the broker restart, containing the unacked message and a new one that is healthy.

Is this behavior expected to be like that? It seems to me "correct" in the way, that RabbitMQ can not know after the broker restart, whether the message processing was completed or not. But what solution would exist than to get that unacked message back into the queue and to heal the system without restarting the consumer process?

1
rabbitmqspring-amqpspring-rabbit
What Spring AMQP version do you use? How about to try the same solution with the latest one: projects.spring.io/spring-amqp ? – Artem Bilan
Also, turn off autoRecoveryEnabled in the client connection factory; Spring AMQP doesn't need it; there have been some bug fixes in this area (fixed in latest code) but turning off auto recovery should solve it too. – Gary Russell
We use spring amqp 1.7.4.RELEASE and auto recovery is disabled automatically by spring – Bruno Haller

1 Answers

0
votes

The RabbitMQ team monitors this mailing list and only sometimes answers questions on StackOverflow.

Is this behavior expected to be like that? It seems to me "correct" in the way, that RabbitMQ can not know after the broker restart, whether the message processing was completed or not.

Yes, you are observing expected behavior. RabbitMQ will re-enqueue the message once it determines that the consumer is really dead. Since your consumer re-connects with what must be the same consumer tag as before, it is up to that process to ack or nack the message.