Flink config custom jar reporter to yaml, and connect host by JMX console

3
votes

I am a learner and need help please.

I have a Java maven project which implements Flink MetricsReporter interface , as my custom reporter to send metrics.

In my local Flink, I was able to:

  1. Add Flink JMX reporter config in local flink-conf.yaml file.
  2. Start flink on port 8789
  3. Use JConsole (JMX console) to connect to remote localhost:8789
  4. Restart flink, see flink metrics under JConsole Beans.

After this Proof of concept that the workflow works, what I want to do is:

  1. I wrote a custom reporter implementing metrics interface, and I built the reporter into a jar, with all necessary dependencies.
  2. I moved the jar under flink lib/ folder, as suggested from The important paragraph of the Reporter documentation

My jar is named x-y-reporter-1.0-SNAPSHOT.jar The pom 

  <groupId>a.b.c</groupId>
  <artifactId>x-y-reporter</artifactId>
  <version>1.0-SNAPSHOT</version>

I updated below config to flink-conf.yaml

metrics.reporters: x-y-reporter
metrics.reporter.x-y-reporter.class: com.b.c.xyreporter.XYReporter
metrics.reporter.x-y-reporter.class.host: localhost
metrics.reporter.x-y-reporter.port: 8789

Question:

  • I want to replace JMX configs to my own reporter configs in flink-conf.yaml. The above config did not work. How would I do that please? What each parameters mean please?
  • I want to see the metrics from JMX console like the JMX-reporter. Is it also through configuring through the port number?

Thank you very much.

1
yamlwildflyjmxapache-flinkmetrics
Here you have all the parameters explained: ci.apache.org/projects/flink/flink-docs-release-1.5/monitoring/…. – Dawid Wysakowicz
@DawidWysakowicz Thank you. But I don't know which one maps to which. I have tried a few, but none of them work to my "x, y" case. :( – sue
I've got a stupid proposal - use the default jmx configuration, just change the class that will be used for the Metrics. metrics.reporters: jmx metrics.reporter.jmx.class: com.b.c.xyreporter.XYReporter metrics.reporter.jmx.port: 8789 – Georgi Stoyanov
@GeorgiStoyanov Thank you very much. The settings in the question are working, I did not realize connecting to local flink process would catch the metrics automatically. :) – sue

1 Answers

0
votes

Checked flink logs after restarting flink with configuration in the question. It turns out that there's an error within my custom class. After fixing the error, and by connect to LOCAL apache-flink process in JConsole, I can see the class under MBean lists. I do not need to connect to remote or localhost.