1
votes

Given two offsets - a start and end offset, or start/end datetime timestamp (equally fine), I want a Kafka consumer to replay all messages within that window.

I have figured out how to reset offset using kakfa-consumer-groups.sh tool to reset the offset based on datetime or offset, but how do I tell the consumer to stop after say replaying for 10,000 messages or 10 minutes?

1

1 Answers

2
votes

There is no configuration or API that lets you stop a KafkaConsumer after a certain amount of processed offsets or time.

You would need to do this programatically by checking the offsets of the ConsumerRecord or having a timer that will stop the Consumer after a certain time.

Instead of using kafka-consumer-groups tool you could also make use of the seek API of the KafkaConsumer to start at a particular offsets for a partition.