How do you configure the subject naming strategy for Confluent Schema Registry?

Question

I'm using Confluent Schema Registry with Avro and I'd like to use multiple schemas in one Kafka Topic.

The default subject naming strategy TopicNameStrategy does not allow for this because it couples the schema subject name to the topic name. It is apparently possible to override this and set the subject naming strategy to RecordNameStrategy or TopicRecordNameStrategy.

Unfortunately, the documentation is not very clear on how or where the subject naming strategy can be overridden.

This document suggests that you can provide the configuration when creating or modifying a topic:

Starting with Confluent Platform 5.5.0, the naming strategy is associated with the topics. Therefore, you now have the option to configure a naming strategy to something other than the default on a per-topic basis for both the schema subject key and value with confluent.key.subject.name.strategy and confluent.value.subject.name.strategy.

From the Confluent CLI, use the --config option to create or modify a topic with the specified naming strategy. For example:

To create a topic that uses RecordNameStrategy for the value:
./bin/kafka-topics --create --bootstrap-server localhost:9092 \
--replication-factor 1 --partitions 1 --topic my-other-cool-topic \
--config confluent.value.schema.validation=true --config confluent.value.subject.name.strategy=io.confluent.kafka.serializers.subject.RecordNameStrategy

When I try this, the kafka-topics application returns the following error:

Error while executing topic command : Unknown topic config name: confluent.value.subject.name.strategy
[2020-10-31 10:17:00,947] ERROR org.apache.kafka.common.errors.InvalidConfigurationException: Unknown topic config name: confluent.value.subject.name.strategy
 (kafka.admin.TopicCommand$)

I've also tried configuring the subject name strategy when writing records to Kafka using kafka-avro-console-producer via property value.subject.name.strategy as this document suggests. However, the console producer ignores this property and creates a new subject for the topic using the default strategy.

kafka-avro-console-producer \
    --broker-list kafka-broker-0.kafka-broker:9092 \
    --property schema.registry.url='http://kafka-schema-registry:8081' \
    --property value.schema='<MYSCHEMA>' \
    --property value.subject.name.strategy='io.confluent.kafka.serializers.subject.TopicRecordNameStrategy' \
    --topic mytopic

Apparently there used to be a way to configure the strategy on the broker itself, but I can't find any documentation states how to do that either.

What is the correct way to configure the naming strategy? Where should it be configured -- on the topic, on the topic producer (kafka-avro-console-producer in this case), or somewhere else?

Additional context:

I'm running everything in Docker using images confluentinc/cp-kafka:6.0.0, confluentinc/cp-schema-registry:6.0.0, and confluentinc/cp-zookeeper:6.0.0
The kafka-topics and kafka-avro-console-producer versions are also set to 6.0.0

Antón R. Yuste Antón R. Yuste · Accepted Answer · 2020-11-12T07:08:32

kafka-avro-console-producer doesn't support to specify the naming strategy yet. You can use other tool like zoe: https://adevinta.github.io/zoe/advanced/avro/

It's something you can define in the producer as property (key.subject.name.strategy and value.subject.name.strategy). You can choose one of the existing strategies or develop your own.

How do you configure the subject naming strategy for Confluent Schema Registry?

1 Answers