Consume message from another queue when routinq keys are used in RabbitMQ

Question

I've defined one topic exchange (alarms) and multiple queues, each with its own routing key:

• allAlarms, with routing key alarms.#: I want this to be used for receiving all alarms in a monitoring application
• alarms_[deviceID], with routing key alarms.[deviceID], where the number of devices can vary at any given time

When sending an alarm from the device, I publish it using the routing key alarms.[deviceID]. The monitoring app, however, only consumes from the allAlarms queue. This leads to the following problem:

The messages in the allAlarms queue have been consumed, while the messages in the remaining queues are ready. Is there a better way of handling messages from multiple consumers? Ideally, I'd like to be able to also send commands back to the devices using the same queues where the devices publish their alarms.

Answer 1

It looks like you have consumers bound to the allAlarms queue but not to any of the alarms_[deviceID] queues.

In AMQP, a single consumer is bound to a single queue by name (and each queue can have multiple consumers bound to it). Messages are delivered to the consumers of a queue in round robin such that for a given message in a queue there is exactly one consumer that will receive the message. That is, consumers cannot listen to multiple queues.

Since you're using a topic exchange, you're correctly routing a single message to multiple queues via the routing key and queue bindings. This means that you can have a consumer for each queue and when a message is delivered to the exchange, each queue will get a copy of the message and each queue will deliver the message to exactly one consumer on each queue.

Thus, if allAlarms is consuming messages, it's because it has a consumer attached to the queue. If any of the alarms_[deviceID] are not consuming messages then they must not have consumers bound to those individual queues. You have to start up consumers for each alarms_[deviceID] by name. That will allow you to also have different consumer logic for different queues.

One last thing:

Ideally, I'd like to be able to also send commands back to the devices using the same queues where the devices publish their alarms.

You don't want to do this using the same queue because there's nothing that will stop the non-device consumers on the queue from picking up those messages.

I believe you're describing RPC over RabbitMQ. For that you will want to publish the messages to the alarms queues with a reply-to header which is the name of a temporary queue. This temp queue is a single-use queue that the consumer will publish to when it's done to communicate back to the device. The device will publish to the alarms exchange and then immediately start listening to the temp queue for a response from the consumer.

For more info on RPC over RabbitMQ check out this tutorial.

Answer 2

I don't think you need any of the queues for the devices - the alarm_[deviceid] queues.

You don't have any consumer code set up on these queues, and the messages are backed up and waiting for you to consume them.

You also haven't mentioned a need to consume messages from these queues. Instead, you are only consuming messages form the alarmAll queue.

Therefore, I would drop all of the alarm_[deviceid] queues and only have the alarmAll queue.

Just publish the alarms through your exchange and route them all to the alarmAll queue and be done with it. No need for any other routing or queues.