Example connector property file to provide standalone Kafka Connect

0
votes

I am starting to play with CDC and Kafka connect

After countless hours trying, I have come to understand the logic

  1. Set Kafka Connect properties (bin/connect-standalone.sh) with your cluster information
  2. Set Kafka Connect configuration file (config/connect-standalone.properties)
  3. Download your Kafka connector (in this case MySQL from Debizium)
  4. Configure connector properties in whatevername.properties

In order to run a worker with Kafka Connector, you need to

./bin/connect-standalone.sh config/connect-standalone.properties

which answers:

INFO Usage: ConnectStandalone worker.properties connector1.properties [connector2.properties ...] (org.apache.kafka.connect.cli.ConnectStandalone:62)

I know we need to run:

./bin/connect-standalone.sh config/connect-standalone.properties myconfig.properties

My issue is that I cannot find any format description, or example of that myconfig.properties field.

【Extra Info】 Debizium configuration properties list:

  1. https://docs.confluent.io/debezium-connect-mysql-source/current/mysql_source_connector_config.html#mysql-source-connector-config
  2. https://debezium.io/documentation/reference/1.5/connectors/mysql.html

【Question】 Where can I find an example of the connector properties?

Thanks!

1
apache-kafkaapache-kafka-connectdebezium
Why not just use connect-distributed if you only want to use one config to start the worker and all the examples you do find use json? - OneCricketeer

1 Answers

1
votes

I'm not sure if I understood your question, but here is an example of properties for this connector :

connector.class=io.debezium.connector.mysql.MySqlConnector
connector.name=someuniquename
database.hostname=192.168.99.100
database.port=3306
database.user=debezium-user 
database.password=debezium-user-pw
database.server.id=184054
database.server.name=fullfillment
database.include.list=inventory
database.history.kafka.bootstrap.servers=kafka:9092 
database.history.kafka.topic=dbhistory.fullfillment
include.schema.changes=true

The original config is the one from the documentation which I converted from json to properties : https://debezium.io/documentation/reference/1.5/connectors/mysql.html#mysql-example-configuration