Deploying Spring Cloud Data Flow Local Server

Question

As a beginner I try to follow the steps described in Getting Started with Manual Installation.

I can download and start the Spring Cloud Data Flow Local Server and the Spring Cloud Data Flow shell.

Welcome to the Spring Cloud Data Flow shell. For assistance hit TAB or type "help".
dataflow:>app register --name http --type source --uri maven://org.springframework.cloud.stream.app:http-source-rabbit:1.2.0.RELEASE
Successfully registered application 'source:http'
dataflow:>app register --name log --type sink --uri maven://org.springframework.cloud.stream.app:log-sink-rabbit:1.1.0.RELEASE
Successfully registered application 'sink:log'

Then I try create a stream with:

dataflow:>stream create --name httptest --definition "http --server.port=9000 | log" --deploy
Created new stream 'httptest'
Deployment request has been sent

And then send some data, which fails:

dataflow:>http post --target http://localhost:9000 --data "hello world"
> POST (text/plain) http://localhost:9000 hello world
> 500 INTERNAL_SERVER_ERROR
> 500 INTERNAL_SERVER_ERROR
{
  "exception" : "org.springframework.messaging.MessageHandlingException",
  "path" : "/",
  "error" : "Internal Server Error",
  "message" : "error occurred in message handler [org.springframework.integration.amqp.outbound.AmqpOutboundEndpoint@20eacb00]; nested exception is org.springframework.amqp.AmqpConnectException: java.net.ConnectException: Connection refused (Connection refused)",
  "timestamp" : 1546968872545,
  "status" : 500
}
Error sending data 'hello world' to 'http://localhost:9000'

I can see from the log file of the log application, that something goes wrong. But as a beginner I do not really have an idea how to go on or fix the problem.

Any ideas?

Sabby Anandan Sabby Anandan · Accepted Answer · 2019-01-08T19:20:40

Looking at this error:

error occurred in message handler [org.springframework.integration.amqp.outbound.AmqpOutboundEndpoint@20eacb00]; nested exception is org.springframework.amqp.AmqpConnectException: java.net.ConnectException: Connection refused (Connection refused)

It appears you do not have RabbitMQ running locally. While the stream is deployed, if you now start a RabbitMQ instance, the applications would automatically recover and connect to it. You will then see a successful deployment of apps, and the POST should work eventually, too.

The message broker is a Spring Cloud Stream requirement. That's how the event-driven microservices communicate with one another through the pub-sub semantics. More details in Spring cloud Stream ref. guide.

In case you're wondering the role of SCDF, here's some background:

SCDF merely is an orchestration service. When the Local implementation of SCDF deploys the stream, the applications in the stream are spawned as standalone Java processes. They are Spring Boot apps. Upon starting, they attempt to autoconfigure with the underlying binder-implementation library the classpath. In your example, the applications you registered are bundled with rabbit-binder, and because they are not configured to connect to an external RabbitMQ cluster, the applications will attempt to connect to the default connection properties (i.e., "localhost" and the default "port").

You either can choose between RabbitMQ or Kafka, or you can customize any of the out-of-the-box applications to communicate with other binder implementations.

Here's an example for when using Apache Kafka as the binder implementation in SCDF.

Deploying Spring Cloud Data Flow Local Server

1 Answers