MassTransit - publish to all consumer instances

4
votes

I am looking for a way for each consumer instance to receive a message that is published to RabbitMQ via MassTransit. The scenario would be, we have multiple microservices that need to invalidate a cache on notification. Pub-Sub won't work in this instance as there will be 5 consumers of the same type as its the same code per service instance, so only one would receive the message in a traditional PubSub.

Message observation could be an option but this means the messages would never be consumed and hang around forever on the bus.

Can anyone suggest a pattern to use in the context of MassTransit?

Thanks in advance.

1
masstransit
If you have competing consumers, you are scaling horizontally. In this case, having a local cache is a bad idea. Not only you have to deal with exotic use cases like you described, but also you populate each cache independently. If you want to scale horizontally - consider having some distributed cache as well. - Alexey Zimarev
@AlexeyZimarev thanks for your commwnt. We have a very complex hierarchical model held locally in volatile memory that doesn't serialise well, so we need to use in-process storage for speed. When this model changes - weekly, all instances need to be notified to rehydrate their local copy from the store. - KnowHoper

1 Answers

9
votes

You should create a management endpoint in each service, which could even be a temporary queue (just request a receive endpoint without a queue name and one will be dynamically generated). Then, put your queue invalidation consumers on that endpoint. Each service instance will receive a unique instance of the message (when Publish is called), and those queues and bindings will automatically be removed once the service exits.

This is exactly how the bus endpoint works, but in your case, you're creating a receive endpoint which can have consumer message type bindings, so that published messages are received, one copy per service.

cfg.ReceiveEndpoint(cfg => { ... });

Note that the queue name is not specified, and will be automatically generated uniquely.