RabbitMQ Request Response with message acknowldgments

3
votes

I am using RabbitMQ with Java and Spring Framework and I have a producer and a consumer, both of which will have multiple instances running, however each request coming to the producer would generate a message that would be handled by a consumer, and that consumer should be able to reply back to the same producer (the exact same one as it is holding the request) with the answer.

So the requirements are:

  • A producer will receive an HTTP request, it will then generate a message and push it to the queue.
  • The message contents can only be processed by one consumer at a time, no other consumer should process the same message.
  • That message should wait to be acknowledged.
  • Any consumer can handle the message, but when the consumer fails the message should return to the queue because we need to be able to recover from this.
  • When done, the consumer should reply to the exact same producer as it is holding the request now.
  • If the producer fails, that request should also fail, so the consumer should check if the producer is still alive before replying if it wasn't available then it should fail.

so:

Request --> Producer1 --> queue --> consumer1

--> Producer1 waiting

--> consumer1 fails

--> message back to the queue

--> consumer2 fetches the message and continues where 1 left off

--> consumer2 finishes and acks the request message and sends the response

--> consumer2 finds that producer1 has failed so it needs to send a failure signal

Currently, all I could think about is: 1. Direct Reply Pattern: but the problem here is that the message has a no ack requirement, but I need to be able to recover from consumer failures 2. The RPC Model: which doesn't address the issue of the producer failure

Is there any other solutions to this?? Am I doing this wrong?

1
javaspringrabbitmqamqpspring-cloud
Your last point is a bit off with the logic how rabbitmq is meant to be used. The producer shouldn't know who will receive the message nor if anyone will. If the producer knows that someone will receive the message, then why bother using a message broker... Just to note that any client that sends a message is a producer, and any cilent that receives is a consumer. Of course one client can be both at the same time, that the RPC example. Also that's why there is no client failure handling (producer or consumer) by rabbitmq – cantSleepNow
@cantSleepNow We need the message broker for recovery, if we use direct calls, then we cannot recover if the consumer failed, and we also will never be able to know which jobs it was handling and failed. – engma
I know this is a Java question, but in .NET there are lightweight message bus tools MassTransit and NServiceBus that handle request/response with RabbitMQ as the message broker. Those 2 tools sit at an abstraction layer above RabbitMQ and facilitate request/response, fire-and-forget and publish patterns, and handle message serialization/deserialization. I don't know if there are any Java equivalents though. – Andy

1 Answers

0
votes

It seems that to achieve the process we have to use Direct-to way but I am still not sure. I will let you know once arrive a solution.