SCDF custom spring cloud streaming source application does not write produced message to Kafka topic

Question

I have created a simple spring cloud streaming "source" & "Sink" applications with the kafka binder and registered the jars in SCDF (Spring Cloud Data Flow).

Created the steam in SCDF ( source-app | sink-app ) and deployed the stream successfully.

When i try to call the source , the message produced from source is not written to Kafka topic. Topic is created with the name of "steam-name.sourceapp-name".i checked through the Kadeck tool.and from the SCDF source log, i see the message is produced from the source app end as well.but the message is not published to the topic.

Dataflow skipper,local dataflow server,zookeeper and kafka running locally.

My source application is simple, code is below:

@GetMapping("send-it")
@SendTo(Source.OUTPUT)
public Message<String> out() {

    final Message<String> test_test =
            MessageBuilder.withPayload("appTestName: " + "demo" ).build();
    System.out.println("Sending message ....");
    return test_test;

}

Used @EnableBinding(Source.class),@RestController on class level.

Ilayaperumal Gopinathan Ilayaperumal Gopinathan · Accepted Answer · 2019-09-19T13:14:11

The creation of the topic with the name "steam-name.sourceapp-name" confirms that the source app is configured by the SCDF correctly. You can try enabling more debug logging in Kafka client as well as Spring Kafka packages to see what's going on when publishing the messages to the topic.

SCDF custom spring cloud streaming source application does not write produced message to Kafka topic

1 Answers