Spring Integration Kafka - Sending a basic String

Question

I am trying to send a basic String payload using Spring Integration Kafka v1.2.1, but it's failing with the following exception:

2015-09-03 11:50:39.729 ERROR 14418 --- [task-executor-3] [                                    ] o.s.integration.handler.LoggingHandler   : org.springframework.messaging.MessageHandlingException: error occurred in message handler [org.springframework.integration.kafka.outbound.KafkaProducerMessageHandler#0]; nested exception is org.apache.kafka.common.errors.SerializationException: Can't convert value of class [B to class org.apache.kafka.common.serialization.StringSerializer specified in value.serializer
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:84)
at org.springframework.integration.endpoint.PollingConsumer.handleMessage(PollingConsumer.java:74)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.doPoll(AbstractPollingEndpoint.java:219)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.access$000(AbstractPollingEndpoint.java:55)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:149)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:146)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller$1.run(AbstractPollingEndpoint.java:298)
at org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:52)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.kafka.common.errors.SerializationException: Can't convert value of class [B to class org.apache.kafka.common.serialization.StringSerializer specified in value.serializer

My configuration XML looks as follows:

<task:executor id="schedule-request-task-executor" pool-size="5" keep-alive="120" queue-capacity="125"/>

<bean id="kafkaStringSerializer" class="org.apache.kafka.common.serialization.StringSerializer"/>

<int-kafka:producer-context id="schedule-request-producer-context">
    <int-kafka:producer-configurations>
        <int-kafka:producer-configuration topic="schedule.requests"
                                          key-serializer="kafkaStringSerializer"
                                          value-serializer="kafkaStringSerializer"
                                          broker-list="${kafka.brokers}"/>
    </int-kafka:producer-configurations>
</int-kafka:producer-context>

<int-kafka:outbound-channel-adapter
        kafka-producer-context-ref="schedule-request-producer-context"
        channel="schedule-request-channel">
    <int:poller receive-timeout="0"
                fixed-delay="100" time-unit="MILLISECONDS"
                task-executor="schedule-request-task-executor"/>
</int-kafka:outbound-channel-adapter>

And I'm sending the message with the following code:

Message message = MessageBuilder.withPayload("PAYLOAD")
                .setHeader("messageKey", "KEY")
                .setHeader("topic", "schedule.requests")
                .build();
scheduleRequestChannel.send(message);

I looked at the samples at https://github.com/spring-projects/spring-integration-extensions/blob/master/samples/ but these seem to be outdated.

I debugged this and it seems that the value in ProducerRecord is converted to a byte[] when it's passed to the serializer for some reason. Not sure why though. — Yohan Liyanage

Yohan Liyanage Yohan Liyanage · Accepted Answer · 2015-09-03T08:10:36

After debugging SI and Kafka classes, I found that this happens because Spring Integration converts the String to a byte[] unless the key-class-type and value-class-type is specified in Producer Configuration.

Here's the updated configuration in case someone is interested.

<int-kafka:producer-context id="schedule-request-producer-context">
        <int-kafka:producer-configurations>
            <int-kafka:producer-configuration topic="schedule.requests"
                                              key-class-type="java.lang.String"
                                              key-serializer="kafkaStringSerializer"
                                              value-class-type="java.lang.String"
                                              value-serializer="kafkaStringSerializer"
                                              broker-list="${kafka.brokers}"/>

Spring Integration Kafka - Sending a basic String

1 Answers