RabbitMQ single active consumer with passive failover consumers

Question

I have an API that can be scaled horizontally. The API will acknowledge the client requests and it needs to dispatch the work to a secondary system. The secondary system needs to process the work on a first come first serve basis, a queue. Also since the secondary system is accessing resources that can be shared only one instance of it can be active at a given time. The secondary system needs to have a fail over mechanism. If the 1st instance of the secondary system goes down another instance needs to take its place.

I was thinking of using RabbitMQ as the queuing mechanism and have multiple consumers connect but have only one active consumer that will process the work. One of the other instances will take up the job of processing messages if the previously active consumer fails to acknowledge messages. Is this possible with RabbitMQ ?

Also is it possible to acknowledge the message only after the job has been completed?

Thank you.

Answer 1

This is possible with new release of RabbitMQ: https://www.rabbitmq.com/consumers.html#single-active-consumer

Answer 2

This isn't directly possible with RabbitMQ, the way you are wanting.

You can have multiple consumers for a single queue. However, RabbitMQ will deliver messages to any of the consumers that are available to do work. In other words, if you have 3 consumers and you send 3 messages to the queue, each one of those consumers will likely get 1 of the messages.

If you truly need active / passive failover for the consumers, you will need to use another system to manage and monitor instances of the consumer.

Regarding acknowledgement of the work being done: yes. Put your queue into acknowledgement mode by setting no_ack to true. This will require you to acknowledge each message from the consumer. You can hold on to the message until the work is done and then acknowledge the message when the work is complete.

See the worker queue example in the RabbitMQ documentation for an example of using acknowledgements, and for a brief discussion of round-robin message dispatching to consumers.

Answer 3

in this case i suggest to duplicate the message over the queues, ex. if u have 3 consumer for a message, it's preferable to build 3 queues and by exchanger duplicate the message over this 3queue.

Also is it possible to acknowledge the message only after the job has been completed?
yes you can move the ack in your code, in your case after the elaboration of the message.

Answer 4

Since it wasn't mention yet, Consumer Priorities might give you enough in this situation. It depends if by your definition the primary consumer failing to acknowledge messages equals what RabbitMQ considers a consumer in the blocked state. There is also a little more discussion about this here.