Confluent Schema Registry Offset Management

Question

We use Confluent Schema Registry as Avro Schema Repo. What I have observed is that there is no Consumer offset management in Zookeeper. Is it because of the way Schema Registry is designed - not to store offset in Zookeeper but in Kafka or is this a standard design pattern in case for compacted logs/topics in Kafka?

Answer 1

The use of the Confluent Schema Registry does not affect in any way how consumer offsets are handled.

In Kafka version 0.9 and higher (I.e. “new consumer” API) the consumer offsets are stored in a special Kafka topic called __consumer_offsets.

The storage of consumer offsets in Kafka is the same for both compacted topics and regular topics.

Answer 2

The new high level consumers don't store the offsets in Zookeeper. You may check this link https://cwiki.apache.org/confluence/display/KAFKA/Kafka+0.9+Consumer+Rewrite+Design