Issue with spring cloud dataflow and Remote Repository: Apps are installed but i can't deploy streams

Question

i'm facing an issue using spring cloud dataflow connected to a remote repository.

I think i managed to connect the dataflow server to the repository correctly because at first i couldn't import apps and now i can

The problem is that when i try to deploy a stream the dataflow server doesn't see the remote repository.

Here's an example to make myself clear

When i try to import a jar that does not exist the import is successful but if i try to open the details from the UI i get:

Failed to resolve MavenResource: [JAR-NAME] Configured remote repositories: : [repo1],[springRepo]

So i guess that the system sees "repo1"

But then when i deploy a stream (with all valid apps) i get:

Error Message = [Failed to resolve MavenResource: [JAR-NAME] Configured remote repository: : [springRepo]]

I followed this: https://github.com/spring-cloud/spring-cloud-dataflow/issues/982

And this: https://docs.spring.io/spring-cloud-dataflow/docs/1.1.0.BUILD-SNAPSHOT/reference/html/getting-started-deploying-spring-cloud-dataflow.html

This is my docker-compose.yml:

version: '3'

services:
  kafka:
    image: wurstmeister/kafka:2.11-0.11.0.3
    expose:
      - "9092"
    environment:
      - KAFKA_ADVERTISED_PORT=9092
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
      - KAFKA_ADVERTISED_HOST_NAME=kafka
    depends_on:
      - zookeeper
  zookeeper:
    image: wurstmeister/zookeeper
    expose:
      - "2181"
  dataflow-server:
    image: springcloud/spring-cloud-dataflow-server:2.0.2.RELEASE
    container_name: dataflow-server
    ports:
      - "9393:9393"
    environment:
      - spring.cloud.dataflow.applicationProperties.stream.spring.cloud.stream.kafka.binder.brokers=kafka:9092
      - spring.cloud.dataflow.applicationProperties.stream.spring.cloud.stream.kafka.binder.zkNodes=zookeeper:2181
      - spring.cloud.skipper.client.serverUri=http://skipper-server:7577/api
      - spring.cloud.dataflow.applicationProperties.stream.management.metrics.export.influx.enabled=true
      - spring.cloud.dataflow.applicationProperties.stream.management.metrics.export.influx.db=myinfluxdb
      - spring.cloud.dataflow.applicationProperties.stream.management.metrics.export.influx.uri=http://influxdb:8086
      - spring.cloud.dataflow.grafana-info.url=http://localhost:3000
      - maven.localRepository=null
      - maven.remote-repositories.repo1.url= [URL]
      - maven.remote-repositories.repo1.auth.username=***
      - maven.remote-repositories.repo1.auth.password=***
    depends_on:
      - kafka
    volumes:
      - ~/.m2/repository:/m2repo
  app-import:
    image: springcloud/openjdk:latest
    depends_on:
      - dataflow-server
    command: >
      /bin/sh -c "
        while ! nc -z dataflow-server 9393;
        do
          sleep 1;
        done;
        wget -qO- 'http://dataflow-server:9393/apps' --post-data='uri=https://repo.spring.io/libs-release/org/springframework/cloud/stream/app/spring-cloud-stream-app-descriptor/Einstein.RELEASE/spring-cloud-stream-app-descriptor-Einstein.RELEASE.stream-apps-kafka-maven&force=true';
        echo 'Stream apps imported'
        wget -qO- 'http://dataflow-server:9393/apps' --post-data='uri=https://repo.spring.io/libs-release-local/org/springframework/cloud/task/app/spring-cloud-task-app-descriptor/Dearborn.SR1/spring-cloud-task-app-descriptor-Dearborn.SR1.task-apps-maven&force=true';
        echo 'Task apps imported'"
  skipper-server:
    image: springcloud/spring-cloud-skipper-server:2.0.1.RELEASE
    container_name: skipper
    ports:
    - "7577:7577"
    - "9000-9010:9000-9010"

  influxdb:
    image: influxdb:1.7.4
    container_name: 'influxdb'
    ports:
      - '8086:8086'

  grafana:
    image: springcloud/spring-cloud-dataflow-grafana-influxdb:2.0.2.RELEASE
    container_name: 'grafana'
    ports:
      - '3000:3000'
volumes:
  scdf-targets:

Ilayaperumal Gopinathan Ilayaperumal Gopinathan · Accepted Answer · 2019-05-23T14:21:49

You need to set the maven remote repository configuration for the Skipper server as well. It is the Skipper server that takes care of handling the deployment request from the SCDF server and hence the Skipper server requires the similar configuration:

      - maven.remote-repositories.repo1.url= [URL]
      - maven.remote-repositories.repo1.auth.username=***
      - maven.remote-repositories.repo1.auth.password=***

Issue with spring cloud dataflow and Remote Repository: Apps are installed but i can't deploy streams

1 Answers