Unable to Run a Composed Task in Spring Cloud Data Flow

0
votes

I am running latest version of SCDF server on Kubernetes cluster. Every time I try to run a composed task, it tries to fetch the application properties for composed-task-runner application and fails to launch the composed task.

First of all, SCDf is trying to pull the properties (metadata) from Spring Maven repo when I am running the server on k8s. my server behind a firewall and it cannot connect to spring maven repo. I already downloaded the composed-task-runner docker image to my local repo and added the composed-task-runner application using UI. Why it still tries to download metadata from Spring Maven repo ? How do I stop it ?

here is the log : 2020-11-21 15:49:07.591 INFO 1 --- [nio-8080-exec-4] o.s.c.d.s.k.DefaultContainerFactory : Using Docker entry point style: exec 2020-11-21 15:49:58.355 WARN 1 --- [nio-8080-exec-6] .s.c.d.s.s.i.TaskConfigurationProperties : org.springframework.cloud.dataflow.server.service.impl.TaskConfigurationProperties.logDeprecationWarning is deprecated. Please use org.springframework.cloud.dataflow.server.service.impl.ComposedTaskRunnerConfigurationProperties.logDeprecationWarning 2020-11-21 15:50:18.427 WARN 1 --- [nio-8080-exec-6] ApplicationConfigurationMetadataResolver : Failed to retrieve properties for resource org.springframework.cloud:spring-cloud-dataflow-composed-task-runner:jar:2.7.0-SNAPSHOT because of ConnectTimeoutException: Connect to repo.spring.io:443 timed out 2020-11-21 15:50:38.522 WARN 1 --- [nio-8080-exec-6] ApplicationConfigurationMetadataResolver : Failed to retrieve properties for resource org.springframework.cloud:spring-cloud-dataflow-composed-task-runner:jar:2.7.0-SNAPSHOT because of ConnectTimeoutException: Connect to repo.spring.io:443 timed out 2020-11-21 15:50:38.572 INFO 1 --- [nio-8080-exec-6] o.s.c.d.s.k.KubernetesTaskLauncher : Preparing to run a container from org.springframework.cloud:spring-cloud-dataflow-composed-task-runner:jar:2.7.0-SNAPSHOT. This may take some time if the image must be downloaded from a remote container registry. 2020-11-21 15:50:38.573 INFO 1 --- [nio-8080-exec-6] o.s.c.d.s.k.DefaultContainerFactory : Using Docker image: //org.springframework.cloud:spring-cloud-dataflow-composed-task-runner:jar:2.7.0-SNAPSHOT

1
spring-cloudspring-cloud-dataflow
Just to add on, I was on SCDF 2.2.4 version before this and it was working fine. We upgraded to latest version last week and started observing this issue. – Sudhakar Master

1 Answers

0
votes

Looks like, the Composed Task Runner docker image can now be set using the environment variable :

  • name: SPRING_CLOUD_DATAFLOW_TASK_COMPOSED_TASK_RUNNER_URI value: 'docker://springcloud/spring-cloud-dataflow-composed-task-runner:2.6.0'

We were on SCDF server 2.2.4 version before this and we had to manually add the composed task runner as an Application using the dashboard UI.

Right now, all I had to do is to download this image and push to my local git repo and use it here.