RabbitMQ: Publishing message when consumer is down and later consumer can't consume message without named queue

0
votes

I have a producer and a consumer. Multiple instances of the consumer are running. When producer publishes a message, my intention is to consume the message by all the instances. So, I am using the direct exchange. Producer publishes a message to the direct exchange with a topic. Consumers are listening to that topic with the exclusive queue. This process is working fine when the consumer is up and producer publishes a message. But when consumers are down and producer publishes a message, consumers are not consuming this message when up.

I googled about the issue. A suggestion was to use named queue. But if I use named queue, messages will be consumed following the round-robin algorithm. That does not meet my expectation to consume the same message by all the consumers.

Is there any other solution?

Appreciated your help.

1
rabbitmqspring-rabbit
Hi, I need more details to help you: Your requirement is to have all consumers consume all the messages, right? In this case, why do they have to consume all the messages, provided that they are the same instance of a consumer? The question is: what is your use case? – cdelmas
Niloy - do you understand what a fanout exchange is? That is probably what you want. – Luke Bakken
@cdelmas, I have two application. Let's say appA, appB. I do some caching in appB. But in appA there are some operation which may change some data which is cached in appB. So, this is done through RabbitMQ instead of REST API. Now, I have multiple instance of appB. So, I want that all the instances cache to be updated. In that case, when appA publishes any messages to the MQ, all the instances of appB must consume the message to update their caches. This is my story. Let me know if you need further information. – Niloy Datta
@LukeBakken, Yes, I understand fanout exchange is. Please check this: stackoverflow.com/questions/53863678/… – Niloy Datta
I don't think you do, because my understanding of your description is that you should use a fanout exchange. If you want the same message to be sent to multiple consumers, you use a fanout exchange and each consumer declares its own (possibly exclusive) queue that is bound to that exchange. The best path forward at this point would be to share an example of what you have tried (in code form) and explain how that is not meeting your goals. – Luke Bakken

1 Answers

0
votes

There are two solutions to your issue.

Using named queue is one of them. Set your exchange in fanout mode and subscribe your named queues to it. Doing so, when a publisher send a message in your exchange, it will be dispatched to all the queues listening.

You can then have one or more consumer for each queue (allowing you to scale). You'll have to define a named queue / consumer. When one consumer disconnect, his queue still receive messages and when he comes back he can consume them.

You should be able to do what you want that way.

The other way is more for your personnal knowledge since you said you want to use RabbitMQ. But in that particular case you could use Kafkha, your consummer could then, after reconnection, resume at the message index he was when he disconnected.

Please update me if it doesn't work :)