KAFKA JDBC Source connector adds the default schema

Question

I use the KAFKA JDBC Source connector to read from the database ClickHouse (driver - clickhouse-jdbc-0.2.4.jar) with incrementing mod. Settings:

    batch.max.rows = 100
    catalog.pattern = null
    connection.attempts = 3
    connection.backoff.ms = 10000
    connection.password = [hidden]
    connection.url = jdbc:clickhouse://<ip>:8123/<schema>
    connection.user = user
    db.timezone = 
    dialect.name = 
    incrementing.column.name = id
    mode = incrementing
    numeric.mapping = null
    numeric.precision.mapping = false
    poll.interval.ms = 5000
    query = 
    query.suffix = 
    quote.sql.identifiers = never
    schema.pattern = null
    table.blacklist = []
    table.poll.interval.ms = 60000
    table.types = [TABLE]
    table.whitelist = [<table_name>]
    tables = [default.<schema>.<table_name>]
    timestamp.column.name = []
    timestamp.delay.interval.ms = 0
    timestamp.initial = null
    topic.prefix = staging-
    validate.non.null = false

Why does the connector additionally substitute the default scheme? and how to avoid it?

Instead of a request

SELECT * FROM <schema>.<table_name> WHERE <schema>.<table_name>.id > ? ORDER BY <schema>.<table_name>.id ASC

I get an error with

SELECT * FROM default.<schema>.<table_name> WHERE default.<schema>.<table_name>.id > ? ORDER BY default.<schema>.<table_name>.id ASC

Where you create the table in default schema or any specific schema? — Sounak Saha
in a specific scheme. I write it in cinnection.url. jdbc:clickhouse://<ip>:8123/<schema> — asanisimov

Sounak Saha Sounak Saha · Accepted Answer · 2021-06-21T14:01:30

You can create CH data source object like below (Where schema name is not passed).

final ClickHouseDataSource dataSource = new ClickHouseDataSource(
        "jdbc:clickhouse://"+host+"/"+user+"?option1=one%20two&option2=y");

Then in SQL query, you can specify a schema name(schema.table). So it will not add the default schema in your query.

KAFKA JDBC Source connector adds the default schema

1 Answers