Does spring cloud dataflow for batch use only require skipper?

0
votes

I'm trying to get a handle on the architecture of spring cloud dataflow. Specifically in using it to orchestrate purely batch pipelines. I've deployed all the components in k8s and noticed I can deploy the skipper component without a middleware (Kafka / RabbitMQ) and run batch jobs successfully through the server component.

Per the docs I can see skipper is used to orchestrate streaming jobs. Is skipper (and subsequently the Kafka/RabbitMQ middleware) a necessary piece of spring cloud dataflow when using for purely batch jobs or tasks?

1
spring-batchspring-cloudspring-cloud-dataflow

1 Answers

1
votes

Yes, that's correct. Spring Cloud Skipper is not needed when you only run the task (batch) applications.

The idea of Spring Cloud Skipper was to provide continuous deployment experience for Streaming applications. The continuous deployment support for task(batch) applications was embedded into Spring Cloud Data Flow itself since SCDF 2.3.x.