1
votes

Flink window documentation says:

Having a keyed stream will allow your windowed computation to be performed in parallel by multiple tasks, as each logical keyed stream can be processed independently from the rest.

Does this statement means that each window is evaluated by a dedicated thread? Will there be 50 million threads in parallel execution if I have 50 million unique keys creating 50 million window?

1

1 Answers

4
votes

No, Flink does not spawn a new thread for each distinct key.

A window operator that is applied on a keyed stream can run in parallel. The input stream is partitioned on the key attribute, such that all records with the same key are processed by the same parallel instance of the window operator. Hence, each parallel operator instance is responsible for processing a subset of the overall key domain.