Push data from Clickhouse to Kafka

Question

For test purposes I have to push data from a Clickhouse-server to Kafka. I already tried to use the Confluent JDBC connector but this doesen't work very well.

It also seems the Clickhouse-Kafka-Engine only works in the direction Kafka -> Clickhouse, so that the Clickhouse-server works as consumer.

Is ther a convenient way to use a Table in Clickhouse as a producer or have I write my own producer?

Answer 1

I'd suggest offloading tasks like this outside ClickHouse. You can dump the testing data via clickhouse-client and pipe it to a kafka client via shell. Check this out https://github.com/fgeller/kt