Why kafka 0.10 console producer cannot send messages to kafka 0.9?

Question

Why kafka 0.10 console producer cannot send messages to kafka 0.9?

I am starting kafka console consumer in version 0.9 (on the server side). I am starting kafka console producer in version 0.10 (on the client side). Then I got in producer exception (below).

How it could be that new producer cannot send messages to old consumer? Is it possible to set new producer in a kind of deprecated mode to send messages to old kafka?

$ bin\windows\kafka-console-producer.bat --broker-list my_server:9092 --topic my_topic

[2016-08-30 14:03:53,365] ERROR Uncaught error in kafka producer I/O thread: (org.apache.kafka.clients.producer.internals.Sender) org.apache.kafka.common.protocol.types.SchemaException: Error reading field 'brokers': Error reading field 'host': Error reading string of length 25452, only 69 bytes available at org.apache.kafka.common.protocol.types.Schema.read(Schema.java:73) at org.apache.kafka.clients.NetworkClient.parseResponse(NetworkClient.java:380) at org.apache.kafka.clients.NetworkClient.handleCompletedReceives(NetworkClient.java:449) at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:269) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:229) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:134) at java.lang.Thread.run(Thread.java:745)

Answer 1

Kafka 0.10.0 introduced a new message format. I'm not completely sure but it looks like you're sending a message with the new format to a broker that clearly knows nothing about that.

You should try to force the 0.10 producer to use the old message format. Here you can find more info: http://kafka.apache.org/documentation.html#upgrade_10_breaking