read filter messages from Kafka topic according to consumer

0
votes

How can we consume custom messages from multiple or a single partition in a Kafka topic?

How can I read only 1 message when I consume messages from multiple partitions in a Kafka topic?

1
apache-kafkakafka-consumer-apiproducer-consumer
What do you mean by custom messages? What's the use case that you are trying to achieve? – Bitswazsky
Try something like kafka-console-consumer --topic foo --bootstrap-servers kafka:9092 --partition=0 --offset=0 --max-messages=1 – OneCricketeer
My mean is to consume message according to consumer need like if consumer want to get only 1 message from partition how can we read it ? – Ishant Kumar

1 Answers

0
votes

You can set the max.poll.records consumer property to set an upper bound on the number of records returned in a single poll.

An alternative (and more general) approach would be to read any number of messages, but only commit the offset up to a certain point, so that a fewer messages get 'consumed'. Remember, in Kafka messages never actually get consumed in the sense that they don't get purged when read — there is an illusion of consumption by persisting the topic-partition offsets on a consumer-group basis.