3
votes

I'm looking for a way to implement aggregation/fold function on a window that also have a state. I understand how to aggregate on a window, and how to use key/global state - but not both.

Just to be clear, when I say a window with state - I mean that the state should be initialized (nullified) every time the window is changed/moved.

For example: I want to count the number of events keyed by event type every 5 minutes. But in addition to event type (which is the window key) the event has some id field - and I would like to count each id only once - so I need to save a state of all the ids I've already counted in that window.

Is there a simple way to do this in Flink?

1

1 Answers

2
votes

Flink has a RichReduceFunction, which will give you access to state that is global across all windows for a given key. If you need per-window state, see [Flink-5929] which will be part of Flink 1.3.