Dead-lettering dead-lettered messages in RabbitMQ

Question

Here's what we have here:

• Topic Exchange DLE, which is intended to be a Dead-Letter Exchange
• Topic Exchange E, which is the "main" Exchange
• Several Queues (EQ1, ..., EQn) bound to E (and initialized with x-dead-letter-exchange = DLE), each with own Routing Key. These queues are the ones being consumed from.
• For each EQn, there's a DLEQn (initialized with x-dead-letter-exchange = E and x-message-ttl = 5000), bound to DLE with the same routing key as EQn. These queues are not being consumed from

What I want is the following: if a consumer cannot process a message from EQn, it Nacks the message with requeue: false and it gets to the DLEQn - that is, to an appropriate queue on the Dead-Letter Exchange. Now, I want this message to sit on the DLEQn for some time and then get routed back to the original queue EQn to be processed again.

Try as I might, I could not get the "redelivery to the original queue" working. I see that messages sit in the DLEQn with all the right headers and Routing Key intact, but after TTL expires they just vanish into thin air.

What am I doing wrong here?

Answer 1

Yes, you can do this. We are currently doing this in production and it works great. The code is too long to include here but I will show you the diagram I created that represents the process. The basic idea is that the First DLX has a TTL, once that TTL expires the message goes into a 2nd queue to be re-sent back into the original.

Answer 2

RabbitMQ detects message flow cycling (E -> DLE -> E -> DLE ...) and silently drops messages:

From DLX manual (Routing Dead-Lettered Messages section):

It is possible to form a cycle of dead-letter queues. For instance, this can happen when a queue dead-letters messages to the default exchange without specifiying a dead-letter routing key. Messages in such cycles (i.e. messages that reach the same queue twice) will be dropped if the entire cycle is due to message expiry.

Answer 3

That post is pretty old, but it took me days to find a solution for a similar problem, so I thought I should share my solution here.

We're receiving messages in TargetQueue (no TTL!!!, bound to TargetExchange) which may be nacked by the consumer. TargetQueue has a DLX defined (RetryExchange), which in turn has bound a corresponding queue (RetryQueue, with a TTL of 60 secs, TargetExchange defined as DLX).

So if the consumer nacks a message from TargetQueue, it gets queued up in the RetryQueue and because of the TTL, the message gets nacked again and requeued in the original TargetQueue. The clue was, that TargetQueue may not have a TTL defined, otherwise a message like this appears in the RabbitMQ log:

Dead-letter queues cycle detected: [<<"TargetQueue">>,<<"RetryQueue">>,<<"TargetQueue">>]

So in the end the solution is pretty straight forward (and only needs one consumer). I got the final inspiration from https://medium.com/@igkuz/ruby-retry-scheduled-tasks-with-dead-letter-exchange-in-rabbitmq-9e38aa39089b