Rereading message from Kafka topic by refusing acknowledgement

Question

I'm implementing Kafka consumer with custom acknowledgement mechanism using spring-integration-kafka.

The code from this example was used.

What I'm trying to achieve is when an exception is thrown, the acknowledgement should not be sent back to Kafka (i.e. no offset commit should be performed) so the next fromKafka.receive(10000) method call will return the same message as the previous one.

But I faced with a problem: even if the acknowledgement isn't sent to Kafka, the consumer knows somehow the offset of the next message and continues to read new messages in spite of the fact that offset value in offset topic remains unchanged.

How to make consumer reread message in case of some failures?

Answer 1

There's not currently any support for re-fetching failed messages

One thing you can do is add retry (e.g. using a request handler retry advice) downstream of the message-driven adapter.

By not acking, the message(s) will be delivered after a restart but not during the current instantiation.

Since messages are prefetched into the adapter, one thing you could do is detect the failure, stop the adapter, drain the prefetched messages and restart.

You could inject a custom ErrorHandler to stop the adapter and signal to your downstream flow that it should ignore the draining messages.

EDIT

There is now a SeekToCurrentErrorHandler.