JDBC Confluent kafka Connector and Topic per schema

0
votes

We recently started using Confluent Kafka-JDBC connector to import RDBMS data.

As part of default configuration settings, it seems that one Topic is created for every Table in the schema.

I would like to know if there is any way to

  1. Create Topic per schema rather than every table. And if Topic per Schema is enabled then can Schema evolution (With Schema Registry) be supported on a table basis ?

  2. If Topic per schema is not possible then are there any guidelines on how to manage hundred's or thousands of topics ? Considering that there will one to one mapping between number of tables to number of topics ?

Thanks in advance,

1
apache-kafkaapache-kafka-connectconfluent-platformconfluent-schema-registry

1 Answers

2
votes

Create Topic per schema rather than every table.

No - it's either n tables -> n topics, or 1 query -> 1 topic.

any guidelines on how to manage hundred's or thousands of topics ?

Adopt a standard naming pattern for them. Use topic-specific configuration as required.