Say we have 3 kafka partitions for one topic, and I want my events to be windowed by the hour, using event time.
Will the kafka consumer stop reading from a partition when it is outside of the current window? Or does it open a new window? If it is opening new windows then wouldn't it be theoretically possible to have it open a unlimited amount of windows and thus run out of memory, if one partition's event time would be very skewed compared to the others? This scenario would especially be possible when we are replaying some history.
I have been trying to get this answer from reading documentation, but can not find much about the internals of Flink with Kafka on partitions. Some good documentation on this specific topic would be very welcome.
Thanks!