Kafka connect JDBC source connector not working

Question

Hello Everyone, I am using Kafka JDBC Source connector using for postgres. Following is my connector configuration. Some how it is not bringing any data. What is wrong in this configuration?

{
    "name": "test-connection",
    "config": {
       "connector.class": "io.confluent.connect.jdbc.JdbcSourceConnector",
       "mode": "timestamp",
       "timestamp.column.name": "TEST_DT",
       "topic.prefix": "test",
       "connection.password": "xxxxxx",
       "validate.non.null": "false",
       "connection.user": "xxxxxx",
       "table.whitelist": "test.test",
       "connection.url": "jdbc:postgresql://xxxx:5432/xxxx?ssl=true&stringtype=unspecified",
       "name": "test-connection"
},
  "tasks": [],
  "type": "source"
}

Do I need to create the topic or does it get generated automatically?

I expect the data to be flowing based on the example but the data is not flowing.Following is the log I see in the kafka connect. But, no data is flowing in.

Log

[2019-07-07 20:52:37,465] INFO WorkerSourceTask{id=test-connection-0} Committing offsets (org.apache.kafka.connect.runtime.WorkerSourceTask) [2019-07-07 20:52:37,465] INFO WorkerSourceTask{id=test-connection-0} flushing 0 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask)

Kingindanord Kingindanord · Accepted Answer · 2019-07-08T14:46:53

Do I need to create the topic or does it get generated automatically?

it generates automatically with the "test" prefix you set in "topic.prefix": "test"

so your topic is called "testtest-connection" or "testtest.test"

it is possible that you are using Avro schema, if so, you have to consume the topic with Avro consumer.

Kafka connect JDBC source connector not working

1 Answers