In our project we are using spring-cloud-stream-binder-kafka version 1.1.2 to integrate with kafka. Recently we had a situation where one of our service consumed old messages(already consumed) from a topic after startup. That topic has 2 partitions and 2 consumers grouped under a consumer group. We are not sure whether the offset is committed to zookeeper properly or not. Below error message is thrown for each messages during the startup.
[-kafka-listener-2] ERROR o.s.k.listener.LoggingErrorHandler.handle - Error while processing: ConsumerRecord(topic = statemachine_deal_notification, partition = 1, offset = 926, key = null, value = [B@6fab0a32)
To make sure this doesn't happen again we always wanted to read only the latest message in the topic. I found setting resetOffsets to true and startOffset to latest will do the job. But these properties doesn't impact the consumer. Later found this feature has been dropped.
Is there any other way that we can make sure consumers in a specific group consumes only the latest messages ???.
