Spring Cloud Data Flow Grafana Prometheus not showing stream data

Question

I launch Spring cloud data flow with docker-compose base on this website.

https://dataflow.spring.io/docs/installation/local/docker/

I created 3 apps, Source, Processor & Sink.

I ran

export STREAM_APPS_URI=https://dataflow.spring.io/Einstein-BUILD-SNAPSHOT-stream-applications-kafka-maven

When I run docker-compose -f ./docker-compose.yml -f ./docker-compose-prometheus.yml up, all my containers start up as specified in the docker-compose.yml and docker-compose-prometheus.yml.

I proceed to register the 3 applications and then create a stream with them.I then run it.

Question 1: The status of the stream is always "Deploying". Shouldn't it be Deployed?

Question 2:

When I click on the "Grafana" icon, it launches the Grafana page but there is no data on it. Why is it so? I am using kafka as my messaging middleware as shown below.

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-stream-binder-kafka</artifactId>
</dependency>

This is my pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.1.7.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>org.dso.info.io</groupId>
    <artifactId>usage-detail-sender-kafka</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>usage-detail-sender-kafka</name>
    <description>Demo project for Spring Boot</description>

    <properties>
        <java.version>1.8</java.version>
        <spring-cloud.version>Greenwich.SR2</spring-cloud.version>
        <prometheus-rsocket-spring.version>0.8.0</prometheus-rsocket-spring.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-stream</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-stream-binder-kafka</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-stream-test-support</artifactId>
            <scope>test</scope>
        </dependency>
        <!-- prometheus-rsocket monitoring -->
        <dependency>
            <groupId>org.springframework.cloud.stream.app</groupId>
            <artifactId>app-starters-micrometer-common</artifactId>
            <version>2.1.1.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>io.micrometer</groupId>
            <artifactId>micrometer-registry-prometheus</artifactId>
        </dependency>
        <dependency>
            <groupId>io.micrometer.prometheus</groupId>
            <artifactId>prometheus-rsocket-spring</artifactId>
            <version>${prometheus-rsocket-spring.version}</version>
        </dependency>
        <dependency>
            <groupId>io.micrometer.prometheus</groupId>
            <artifactId>prometheus-rsocket-client</artifactId>
            <version>${prometheus-rsocket-spring.version}</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-cloudfoundry-connector</artifactId>
        </dependency>
    </dependencies>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <repositories>
        <repository>
            <snapshots>
                <enabled>true</enabled>
            </snapshots>
            <id>spring-snapshots</id>
            <name>Spring Snapshots</name>
            <url>https://repo.spring.io/snapshot</url>
        </repository>
    </repositories>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

What do you see when you go to your applications' /metrics page? (Or whatever endpoint they are configured to export Prometheus metrics to.) If the relevant stream metrics are there (you can check what those metrics are by clicking Edit on one of the Grafana panels and looking at the query), then the problem is on the Prometheus/Grafana end of things. If the metrics are not there (or the /metrics page is missing altogether) then it's on the application end. — Alin Sînpălean

Vinicius Miqueloti Vinicius Miqueloti · Accepted Answer · 2020-05-04T16:25:26

I solved this issue adding some dependencies and some entries on properties.

pom.xml:

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-webflux</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud.stream.app</groupId>
            <artifactId>app-starters-micrometer-common</artifactId>
            <version>2.1.2.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>io.micrometer</groupId>
            <artifactId>micrometer-registry-prometheus</artifactId>
        </dependency>
        <dependency>
            <groupId>io.micrometer.prometheus</groupId>
            <artifactId>prometheus-rsocket-spring</artifactId>
            <version>0.9.0</version>
        </dependency>

application.properties:

spring.cloud.dataflow.applicationProperties.stream.management.metrics.export.prometheus.enabled=true
spring.cloud.dataflow.applicationProperties.stream.management.metrics.export.prometheus.rsocket.enabled=true
spring.cloud.dataflow.applicationProperties.stream.management.metrics.export.prometheus.rsocket.host=prometheus-rsocket-proxy
spring.cloud.dataflow.applicationProperties.stream.management.metrics.export.prometheus.rsocket.port=7001
spring.cloud.dataflow.applicationProperties.stream.management.endpoint.health.show-details=always
spring.cloud.dataflow.applicationProperties.stream.management.endpoint.health.show-components=always
spring.cloud.dataflow.applicationProperties.stream.management.endpoints.jmx.exposure.include=*
spring.cloud.dataflow.applicationProperties.stream.management.endpoints.web.exposure.include=*

Spring Cloud Data Flow Grafana Prometheus not showing stream data

1 Answers