RabbitMQ Mirrored Queues on Multiple Clusters

0
votes

Is it possible to use RabbitMQ HA using multiple(2) RabbitMQ clusters?

Here is my requirement:

We have 2 RabbitMQ clusters (each with 4 nodes). All the nodes in both the clusters will be using same Erlang cookie. So that, even though these 2 clusters are physically in separate locations, but will act as a single cluster with 8 nodes.

We are planning to use HAProxy to load balance both the clusters (8 nodes). Both publisher and consumer will be using this proxy to connect to the broker.

We would like to use mirrored queues for HA with ha-mode:exactly, ha-params:4, ha-sync-mode:automatic along with auto-heal for cluster_partition_handling.

Question:

  1. In case of HA, is there a way we can specify to use 2 nodes from the first cluster and 2 nodes from the second cluster. As I understand, this can be done via policy ha-mode:nodes and use node names but that way it will always use the same node, can this setup be dynamic?

  2. As both the clusters are very reliable, will it be the right approach to use auto-heal for cluster_partition_handling (in case of split brain)?

  3. As per this "By default, queues within a RabbitMQ cluster are located on a single node (the node on which they were first declared). This is in contrast to exchanges and bindings, which can always be considered to be on all nodes.". Does this mean exchanges are mirrored by default? So when a message arrives at an exchange and that node goes down, will the message be available on the other exchange on the other node?

1
rabbitmqrabbitmq-exchangerabbitmqctl

1 Answers

0
votes

The RabbitMQ team monitors this mailing list and only sometimes answers questions on StackOverflow.

So that, even though these 2 clusters are physically in separate locations, but will act as a single cluster with 8 nodes.

Please do not do this. RabbitMQ clusters require reliable network connections with low latency. If your cluster crosses a WAN or availability zone your chance of having network partitions greatly increases. See this section of the docs for more information. You should use either the shovel or federation feature.

Does this mean exchanges are mirrored by default? So when a message arrives at an exchange and that node goes down, will the message be available on the other exchange on the other node?

Yes and yes.