Error when use kafka-avro-console-consumer consume avro message sent by spring cloud stream kafka

1
votes

I tried official schema-registry-confluent example (consumer/producer) with my local installed confluent 4.0.0, it can send "sensor" avro message when send post request and receive in listener, but when I use kafka-avro-console-consumer tool shipped with confluent 4.0.0 to view avro message sent, the tool raised the following error (a). I also tried to consume avro message sent by kafka-avro-console-producer tool (and the message can be shown correctly in kafka-avro-console-consumer tool) , it reported the following error (b). Doesn't the given example support confluent 4.0.0? Many thanks!

(a) kafka-avro-console-consumer error =>

ERROR Unknown error when running consumer: (kafka.tools.ConsoleConsumer$:107) org.apache.kafka.common.errors.SerializationException: Error deserializing Avro message for id -1 Caused by: org.apache.kafka.common.errors.SerializationException: Unknown magic byte!

(b) listener error =>

org.springframework.messaging.converter.MessageConversionException: Could not read JSON: Invalid UTF-32 character 0x51473863 (above 0x0010ffff) at char #1, byte #7); nested exception is java.io.CharConversionException: Invalid UTF-32 character 0x51473863 (above 0x0010ffff) at char #1, byte #7) at org.springframework.messaging.converter.MappingJackson2MessageConverter.convertFromInternal(MappingJackson2MessageConverter.java:234) ~[spring-messaging-5.1.0.BUILD-SNAPSHOT.jar!/:5.1.0.BUILD-SNAPSHOT]

1
avrospring-cloud-streamconfluent-platformconfluent-schema-registry

1 Answers

0
votes

Every time there is a MessageConversionException it simply means that the provided Message Converters could not convert Message. . . and rightfully so as we can't handle every scenario, only the common ones. So, you should implement custom Message Converter (either from scratch or extending one of the existing ones). Here is more info on how to define custom Message Converter https://docs.spring.io/spring-cloud-stream/docs/Fishtown.M1/reference/htmlsingle/#spring-cloud-stream-overview-user-defined-message-converters