Apache Flink, get last event in the window

Question

I'm working on a project where i have a window with a size of 4 days, with a step of 1 day

.timewindow(Time.days(4), Time.days(1))

and i have also a trigger

.trigger(new myTrigger)


onEventTime ---> Continue
onProccessingTime ---> Continue
clear ---> Purge
onElement---> (if element.isFinalTransaction) TriggerResult.FIRE_AND_PRUGE

isFinalTransaction is a boolean, when true it call FAP. the mean question is how can i make it return true/false depending on if the element is the last in the window or not

is there any method that can tell us if the current element is the last one in the window? is there any method that can tell us if the current window is done (before sliding) or not ?

Sorry I didn't get the question at all. Why do you actually need a custom trigger for that. That is actually what the default trigger should do. Or do you have a special definition of 'FinalTransaction'? — TobiSH
@TobiSH well, FinalTransaction is a boolean, that can tell the trigger if the actual event is the last one in the window — TheEliteOne
I think @TobiSH's question is that it seems like the regular windowing support is all that you need. So if you could provide more details as to WHY you need a custom trigger, that would help. — kkrugler

Garrett Thomas Garrett Thomas · Accepted Answer · 2018-08-01T13:30:25

From the abstract trigger class (https://github.com/apache/flink/blob/master//flink-streaming-java/src/main/java/org/apache/flink/streaming/api/windowing/triggers/Trigger.java)

The short answer is no. The method onElement is called for every element that gets added to the pane. When an element gets added it's impossible to know if it is the last element, because that information is not known until the next element comes (and we see if it was in this window or the next one).

However, one alternative would be to check if the element is sufficiently close to the end of the end of the window (because onElement has access to window e.g. if (timestamp > window.getEnd - delta) ...

However, I can not think of a use case in which I would recommend this. If you need access to the last element in the window, you should probably just use a WindowFunction and in the apply method get the last element of the input iterable (input.last).

Apache Flink, get last event in the window

1 Answers