Flink Confusion About Event Time Windows in Relation to First Item's Timestamp

Question

I have a log of records with data and timestamps, and the records are received in order of ascending timestamp by my Flink application. After the first item from a certain key arrives to the window, I want to close the window after X event time and check whether enough items arrived for some condition, and emit a pass or fail for that key.

Is this impossible with the basic window functions in Flink? For example, if I want my windows to be 30 seconds long, but the first item from a key arrives at 15 seconds and the last arrives at 40 seconds, it seems that the window will close at 30 and the trail of records for that key will be split into two windows. In this situation what I wanted was the window to start at 15 seconds in event time and close at 45.

Answer 1

In my experience these use cases are much easier implemented in a KeyedProcessFunction than with a custom window (the built-in windows don't work for your requirements):

• in processElement you can buffer elements into managed state and register a timer for some time in the future (30 seconds)
• in onTimer you check whether enough elements have arrived, evaluate your condition and emit downstream.