How to get the latency number of flink job

Question

I know flink collects latency metrics out of box, but I want to know how can I get the latency data of my job.

Do I have to use custom reporter(e.g. graphite) to view the latency?

I saw there is latency metrics in flink dashboard, but no data when I added the latency of a sink operator. If this is not the latency data, what is it?

Thank you.

I am using Flink V1.2.1 by the way.

David Anderson David Anderson · Accepted Answer · 2017-07-28T08:28:11

The Flink dashboard can only display single values. But you can examine the latency by using the REST api. The URL should look like this:

http://localhost:8081/jobs/f184a26ee033d5448042c18de57499a1/vertices/91f34222e409e97879611b1fd334816e/metrics?get=0.Sink__Unnamed.latency

And the reply will look like this:

[
  {
    id: "0.Sink__Unnamed.latency",
    value: "{LatencySourceDescriptor{vertexID=1, subtaskIndex=0}={p99=416.0699999999999, p50=253.0, min=102.0, max=421.0, p95=378.04999999999995, mean=249.953125}}"
  }
]

You can use http://localhost:8081/jobs/ to find your job id, and then http://localhost:8081/jobs/<job-id>/vertices/ to choose an operator in your execution graph. http://localhost:8081/jobs/<job-id>/vertices/<operator-id>/metrics will show the available metrics.

How to get the latency number of flink job

1 Answers