0
votes

I have created a simple Debezium/MySQL connector:

{   "name": "debezium_source_books",   
    "connector.class": "io.debezium.connector.mysql.MySqlConnector",   
    "tasks.max": "1",   
    "database.hostname": "...",   
    "database.port": "3306",  
    "database.user": "root",   
    "database.password": "root",
    "database.server.id": "...",
    "database.server.name": "...", 
    "database.history.kafka.bootstrap.servers": "localhost:9092",   
    "database.history.kafka.topic": "strapi",
    "table.whitelist": "strapi.books",

    "include.schema.changes": "true",

    "key.converter": "io.confluent.connect.avro.AvroConverter",   
    "key.converter.schema.registry.url": "http://localhost:8081",

    "value.converter": "io.confluent.connect.avro.AvroConverter",   
    "value.converter.schema.registry.url": "http://localhost:8081" }

It gets the data from the database, but the value is presented as binary and the key is empty.

After running kafka-console-consumer --bootstrap-server localhost:9092 --topic readology.strapi.books --property print.key=true --from-beginning, I get record similar to this:

enter image description here

Schema Registry responds to http requests and the connector did create both -key and -value schemas. Inside the binary data, I can recognise the typical "before" and "after" content.

I get similar results using Kafka's consumer and Landoop's UI, so I don't think it's something related to the consumer.

Debezium created a topic for storing DDL statements, it is displayed correctly.

Do you have any idea what might be wrong? Or, how can I debug it?

1
It seems that kafka-avro-console-consumer shows the correct data. I think the UI from lensesio/fast-data-dev has gone berserk. - user3429660

1 Answers

0
votes

This issue happens because of debezium stores the messages on Avro format on topics. Easily you can consume it with kafka-avro-console-consumer