RabbitMQ Topic exchanges: 1 Exchange vs Many Exchanges

22
votes

I have a scenario where I have a series of processes I need to perform, each step is done and scaled in independent applications. I am using topic exchanges for all exchanges. Current topology is something like this:

P -> X -> Q -> C/P -> X -> Q -> C

We are "versioning" our queues to deal with probable requirements changes effecting message structure. Bindings might look something like this:

step1.exchange bound to step1.v1.queue with binding key step1.v1

step1.exchange bound to step1.v2.queue with binding key step1.v2

There are other binding patterns that are not version related that also make topical exchanges the appropriate choice. However we could get away with only using one exchange to accomplish the same thing.

TLDR: Is their a benefit to using multiple topical exchanges instead of one topical exchange when your use case could work either way?

2
rabbitmqamqp
Thanks @robthewolf. Using separate exchanges certainly helps to conceptualize, explain and document the topology. It does of course require a bit more setup. I am very interested in performance impact as well though. – Alan Peabody

2 Answers

14
votes

I simply copy some key fragments for you.
https://spring.io/blog/2011/04/01/routing-topologies-for-performance-and-scalability-with-rabbitmq/

  • If you have a finite domain of routing keys in an application’s graph then many fanout exchanges might be the right fit (1:1 mapping of exchange per routing key)

  • If you have a potentially infinite number of routing keys, consider topic exchanges

  • For topic routing, performance decreases as the number of bindings increase

  • Fanout exchanges are very fast because they have no routing to process yet if bound to a large number of queues that changes

  • Direct exchanges are a faster form of topic exchanges, provided you do not need the wild card

  • Troubleshooting problems across 100,000+ queues could be tedious versus a topology with more bindings, fewer exchanges and queues

  • A very high number of exchanges and queues take up more memory which may be significant but this really depends

As of RabbitMQ 2.4.0, released March 23, 2011, a new topic routing algorithm optimization is available that is 60 times faster at peak than the previous topic algorithm. Due to this, one recommendation is to go for less exchanges and queues, and more routing because the time increase is now minimal

7
votes

Take a look at "Routing Topologies for Performance and Scalability with RabbitMQ" http://blog.springsource.org/2011/04/01/routing-topologies-for-performance-and-scalability-with-rabbitmq/