RabbitMQ: How to combine a task queue and a fanout/routing/topic models?

Question

I have an environment with one producer and a number of consumers.
The producer creates 2 types of messages:

1. A message that needs to be processed by ONE consumer only (any consumer will do).
2. A message that needs to be processed by ALL consumers.

How can this be implemented?
For message type 1, a work queue is the suitable model.
For message type 2, a fanout/direct/routing/topic are suitable.
But how do I combine them?

Answer 1

RabbitMQ is very flexible, you could have many different exchange and queue design solutions to meet your requirement.

But, first, we need to understand the relationship and basic rule between queues and consumers:

• If you want a message type to be consumed by only one of all consumers, as you said, you need a worker queue, all the consumers should subscribe to it.
• If you want to a message type to be consumed by each of the consumers, you need to have queues for each consumer, and each consumer only subscribe to its own queue.

When number of queues are clear based on the above understanding. Things left is how to route your messages to these queues. There will be many solutions. Below are just some examples.

One workable solution is to create two exchanges, one for each message type.

| message type | exchange name | exchange type | bound queues      |
|------------------------------------------------------------------|
| type_1       | exchange1     | fanout        | shared_queue      |
| type_2       | exchange2     | fanout        | queue1,queue2,... |

Another workable solution is, if you want to have only one exchange for publishing the two message types to, use 'direct' exchange type:

| routing_key | binding_key | bound queues      |
|-----------------------------------------------|
| type_1      | type_1      | shared_queue      |
| type_2      | type_2      | queue1,queue2,... |

One exchange could have multiple queues bound to it with same binding key. So, when publishing message type 1 with a publish routing key - "type_1", only the shared_queue will receive the message; when publishing message type 2 with publish routing key - "type_2", all the queue1,queue2,... will receive the message.

Use different binding keys for each messages might not be ideal for real cases in case you have more message types and you don't want to use the same routing keys. If so, you might want to use "topic" exchange type instead:

| routing_key | binding_key | bound queues      |
|-----------------------------------------------|
| type_1.1    | type_1.*    | shared_queue      |
| type_2.2    | type_2.*    | queue1,queue2,... |

Answer 2

Direct vs. fanout routing are properties of RabbitMQ exchanges. All RabbitMQ messages are published to exchanges, rather than directly to queues--when you create and use a queue without explicitly creating an exchange, you are actually using a pre-declared default exchange.

You can create multiple exchanges on a single RabbitMQ broker, and you can bind a single queue to more than one exchange. If you want workers to use the same queue for both message types, you can create a direct exchange (for message type 1) and a fanout exchange (for message type 2) and bind each queue to both exchanges. Otherwise, you can create separate queues for each exchange type.

RabbitMQ's AMQP concept guide has a good explanation of exchanges and queues, and Tutorial 3 on the RabbitMQ Getting Started page shows you how to create and bind exchanges.