Waiting for all rabbitmq responses on a fanout exchange?

4
votes

I've configured a rabbitmq fanout exchange called "ex_foo" for a RPC workload. When clients connect to the server, they create their own non-durable RPC receive queue and connect to it with a BasicConsumer. The apps listen for messages/commands and respond to the queue defined in the reply_to part of the request.

One of the simple messages/commands I'm sending out the the fanout exchange (and thus, every application/client connected to it) is a type of ping request message, and my problem is that I don't know how many ping responses I will get (or should expect), because I don't know how many clients are connected to the fanout exchange at any one time. All clients connected to the fanout exchange should reply.

If gets delivered to 10 queues on the fanout exchange (ie: 10 clients are connected), how do I know how many responses to expect? In order to know that, would I have to know how many times it was delivered? Is there anything more sophisticated and a sleep timer? Simply, my admin tool can't just wait indefinitely and needs to quit after it has recveived all pings (or a time-out has elapsed).

1
rabbitmqamqp
Can you treat the ones that don't respond as "down"? – Enno Shioji
Sure, if a node does not respond it's down. But one ping request will generate many ping replies and if I just sleep() for 10 seconds and see how many responses I get, it seems bad, if every node responds within 1 second for example. – xconspirisist
I don't feel it's that bad; you wait for a certain time and if it hadn't responded within that time, it's down <- sounds sensible to me.. – Enno Shioji
I think I asked the same question today. (I marked mine as duplicate as best I could) : stackoverflow.com/questions/37117670/… – granadaCoder

1 Answers

6
votes

What you are looking for is something like a Scatter-Gather (http://www.eaipatterns.com/BroadcastAggregate.html) pattern, isn’t it?

You don’t know the consumers bound to the fan-out, so you can:

  1. implement an keep-alive from the consumer(s) using for example an queue where the producer is bound. Each consumer sends a keep-alive each one second, if you don’t receive a message you can considerer the consumer off-line.
  2. Use an in-memory database where the consumer are registered (always with a keep-alive).
  3. Use the HTTP API to know the consumers list bound to the fan-out, in this way:

http://rabbitmqip/vhost/yourfanout/bindings/source and the result is like this:

 [{"source":"yourfanout","vhost":"/","destination":"amq.gen-xOpYc8m10Qy1s4KCNFCgFw","destination_type":"queue","routing_key":"","arguments":{},"properties_key":"~"},{"source":" yourfanout","vhost":"/","destination":"myqueue","destination_type":"queue","routing_key":"","arguments":{},"properties_key":"~"}]

Once count the consumers you know the replies count.

Call the API before send a request.

NOTE the last-one can works only if you use a temporary queue bound to the consumers.

I found this resource that could help you (http://geekswithblogs.net/michaelstephenson/archive/2012/08/06/150373.aspx)

I don't know exactly your final scope, but with a keep-alive you can wait max one second before decide if the consumer is alive.