rabbitmq multiple consumers on a queue- only one get the message

11
votes

I implemented multiple consumers, who are fetching messages from a single queue, I am doing this using something similar to this example, except that I'm doing basic.get on an infinite loop for polling.

Any idea how do I prevent racing between all consumers, in that only one consumer will get the message and the other will continue to do polling until another message comes?
I tried implementing a logic in which as soon as I get the message I ack it for the message to be removed, but it seems that some other queue managed to get the message before the first queue has ack and removed it. So each consumer got the message.

Thanks in advance

1
pythonrabbitmqamqppika
How are you publishing the message on the queue? How are you consuming it? Do I understand correctly that all your consumers receive the message, and not just one - but you want only one client to receive it? Or do only one client receive the message, and you want everyone to receive it? – MatsLindh
Publishing using channel.basic_publish -direct with routing_key consuming it using basic_get, exactly like in the example above – JavaSa
All my consumers receive the message , and I want that the first who got it will process it, and that only one consumer will get every message from the queue, the one who is available – JavaSa

1 Answers

13
votes

Any idea how do I prevent racing between all consumers, in that only one consumer will get the message and the other will continue to do polling until another message comes?

you can't, the way you have things set up. RabbitMQ will round-robin the messages to the consumers, but only one consumer will receive the message from the queue. This is by design in RabbitMQ, when you have multiple consumers on a single queue.

If you need all consumers to receive all messages, then you need to change your configuration so that each consumer has it's own queue. Then you need to publish your message through an exchange that will deliver the message to all of the queues for all of the consumers.

The easiest way to do this is with a Fanout exchange type.