Apache Flink, Event Time Aggregation - does Watermark associated with Aggregation Key?

0
votes

When we do Aggregation based on Event Time, with a live stream of data - will the watermark for an event with certain key trigger the window operation on windows for other keys?

E.g aggregating on event-time-window of 10 Minutes. Received following event sequence:

id 1 at 12:00
id 1 at 12:08
id 2 at 12:15
id 1 at 12:20

Will event with id 2 fire time window for id 1 12:00-12:10? Or it will only happen at 12:20 when next event with id 1 arrives?

1
apache-flinkflink-streaming

1 Answers

3
votes

Watermarks are not associated with keys -- they are per-stream, or in the case of Kafka, they can be per-kafka-partition -- so yes, a Watermark generated based on an event for one key will trigger windows for other keys as well.

Per-key watermarking is something that has been discussed in the Flink community, and might happen some day, but as of Flink 1.5, that's the situation.