Keep message in topic until all consumers are done with processing

Question

Team, We have a publisher which publishes message to request topic and assume we have 2 consumer instances are running. These 2 instances will acquire message from topic. one consumer processed with out any issues but other consumer while processing crashed in middle. Since it is non-durable topic message is already removed from request topic while consumer acquires the message. Now we dont want that to happen.Unless both of the consumers are done message should be available in topic. We are not able to use durable subscriber for this case as consumer instances are dynamic. Today we have 2 consumers instances and tomorrow based on load we may add 2 more consumer instances and if we have durable subscriber we cannot maintain dynamic clientId and durableSubscriptionNaem. We use camel route for consumer and publisher.

Any approach or option in activemq to handle this?

Thanks, Golden eagle team.

Answer 1

Problem is that this is not how Topics work, if the subscriptions are not durable and the client is not present at the time of publication the message is not retained. What you are looking for sounds more like a Queue so you may want to change your thinking about what your applications are doing.

ActiveMQ has a feature called retroactive consumers that can hold some past Topic messages in memory but this is not reliable as your client can come along late enough that the oldest messages are dropped from memory, or your broker might have failed and been restarted in which case those messages are gone.