Kafka Connect: how can I use the AvroConverter without schema registry?

3
votes

In our worker-distributed props, we have:

value.converter=io.confluent.connect.avro.AvroConverter
value.converter.schema.registry.url=<dev-schema-url>

We've noticed that the performance is bad when we use schema registry. Can I use AvroConverter without the schema registry? I tried https://github.com/farmdawgnation/registryless-avro-converter but we run into:

java.io.IOException Not a data file

  1. Is schema registry contacted with every message the kafka connector reads?

  2. Is the schema cached?

  3. What are the other options to use AvroConverter without the schema registry?

1
apache-kafkaavroapache-kafka-connectconfluent-schema-registry
Maybe you should show your configuration with RegistrylessAvroConverter if you're having problems with that – OneCricketeer
Did you find a solution to the problem? I also get the same exception. – Shikkou

1 Answers

0
votes

We've noticed that the performance is bad when we use schema registry

Have you analysed where the performance problem lies and what causes it? As in, is it network traffic? Response time from the Schema Registry? etc.

There are many, many people using Schema Registry in Production successfully; performance is not a known problem with Schema Registry.

Is schema registry contacted with every message the kafka connector reads?

No

Is the schema cached?

Yes.

What are the other options to use AvroConverter without the schema registry?

None; the AvroConverter requires the Schema Registry.

References: