Naming operators, sources, sinks and patterns to be shown in Flink Execution Plan UI

1
votes

I just want to give names to operators and sources. Let's take an example

enter image description here

This is a screenshot of the execution plan, that I have taken form Flink Dashboard. Here I have 2 sources of DataStreams and then I am joining them. My question is that, can I name these sources as EcgStream and Sp02 Stream for example and Join as Join1?

Reason I am asking this question is because it makes visualization easier. Also, as I was going though Opsclarity page , at the end of the page, they have mentioned the following

Note the task_name and operator_name have been compressed so we can still distinguish the tasks and operators correctly when latency is aggregated across tasks and operators. But these compressed names will not match what is seen in the Flink UI, which will show a fragment of Scala code as the operator name. If you need these names to be meaningful in the metric system, you should supply names in the Flink code of your application. This compressed value only happens for those very long default names that wouldn’t otherwise be legal metric values.

I have another question as well , which is that when ever I make a pattern for CEP , the execution plan UI just show it as a pattern. Is there any way it show what that pattern is like A B+ C? D . Also in case we have multiple patterns we should be able to name than as Patterns{1..n}

3
apache-flinkflink-streamingflink-cep

3 Answers

2
votes

For assigning better names to your operators, see the documentation. This is something you should do not just because it makes the execution plan more readable, but also because it will make your savepoints more robustly restorable as your application evolves (docs).

2
votes

As suggested by @alpinegizmo , I have added UID to source stream as follows

// getting RR interval stream
DataStream<RRIntervalStreamEvent> rrIntervalStreamEventDataStream = envrionment.addSource(new RR_interval_Gen()).uid("RR interval stream");

// getting QRS interval stream

 DataStream<qrsIntervalStreamEvent> qrsIntervalStreamEventDataStream = envrionment.addSource(new Qrs_interval_Gen()).uid("qrs Interval stream");

But execution Graph dont show these uid's

enter image description here

Also the result of below sout is 3

  System.out.println("id for stream 1 is " + stream1.getId());
2
votes

This issue was solved by using name() instead of uid() as follows

// getting RR interval stream
DataStream<RRIntervalStreamEvent> rrIntervalStreamEventDataStream = envrionment.addSource(new RR_interval_Gen()).name("RR Interval stream");


System.out.println("getting transformation for stream 1 = " + rrIntervalStreamEventDataStream.getTransformation());
// getting QRS interval stream

DataStream<qrsIntervalStreamEvent> qrsIntervalStreamEventDataStream = envrionment.addSource(new Qrs_interval_Gen()).name("qrs Interval stream");

The output graph is shown below

enter image description here