RabbitMQ, docker, single queue, multiple consumers

Question

We are using Docker, Spring and RabbitMq in our system. I'm new to AMQP and I'm trying to figure out how the exchanges, queues, bindings and connections work in our system. We have multiple dockerized applications.

In short, when the applications start, they each start to listen to the same queue, and I don't understand how they all can expect to receive the same message. There were some similar questions in stackoverflow that gave me impression that our current system is possibly flawed.

In more detail:

When the docker containers start, multiple applications in different containers use RabbitAdmin to

declare the same exchange: rabbitAdmin.declareExchange(exchange)
declare the same queue: rabbitAdmin.declareQueue(queue)
bind those together: rabbitAdmin.declareBinding(BindingBuilder.bind(queue).to(exhange).with("theSameKey");

They do this because they want to listen to the same messages. As far as I understand, only one of the dockerized programs manages to create the exchange and the queue, and the rest of them try but that has no effect.

After that, each of these applications create and start SimpleMessageContainers for the queue:

simpleMessageContainer.setMessageListener(messageListener)
simpleMessageContainer.addQueueNames(queue.getName())
simpleMessageContainer.start()

Using rabbitmqctl and rabbitmq's web interface, I can see that a single queue has multiple consumers on different channels, corresponding to different docker containers.

Isn't it so that the messageListener resides within the application but RabbitMq creates a Consumer for the queue at the broker when addQueueNames is called, and this Consumer then forwards messages through the connection to the application local messageListener?

Since multiple applications within different docker containers do that same, there are several Consumers for the same queue, just like I'm seeing with rabbitmqctl.

What I don't understand is that doesn't RabbitMq pass messages that end up in the queue to the channels/consumers in a round robin fashion, so that only one of the dockerized application will receive it? The exhanges are of direct and topic type with no fan-out exchanges. If all the dockerized applications wanted to receive the same message, shouldn't then all of them create an own queue for the same exhance with an own queue name but with the same routing key?

I fail to see how the current implementation possibly works properly.

Arun Karunagath Arun Karunagath · Accepted Answer · 2018-05-05T18:43:56

Every container (Actually Apps running inside) try to declare exchanges and queues. Once declared, all other declare commands has no effect (even if parameters are different).

Doesn't RabbitMq pass messages that end up in the queue to the channels/consumers in a round robin fashion.

That is correct. Though you could influence it with prefetch_count and message ack.

If all the dockerized applications wanted to receive the same message, shouldn't then all of them create an own queue for the same exchange with an own queue name but with the same routing key?

Yes, that is the only way because, as per AMQP protocol messages are load balanced between consumers. So if the same message to be processed by all the containers (consumers), they each should have different queue connected to same exchange (direct routing will do).

It is true whether consumers are running using docker or otherwise.

RabbitMQ, docker, single queue, multiple consumers

2 Answers