How does parallelism work in Apache Flink?

Question

Consider I have a Flink cluster of 3 nodes. One node is for Job Manager and the other 2 nodes are for task manager. Each task manager has 3 task slots. So, when I submit my job with parallelism equal to 2, Flink will assign two task slots. So, my question is, how Flink will assign these task slots?

Some scenario

Does Flink assign one task slot from each task manager?

Is there a possibility that both task slots get assign from the same task manager? If yes, my job will not be running if that particular node is down for some reason. How can I avoid downtime in this scenario?

Answer 1

Since Flink 1.10 you can use the configuration setting cluster.evenly-spread-out-slots: true to cause the scheduler to spread out the slots evenly across all available task managers. Otherwise it will use all of the slots from one task manager before taking slots from the other.

Answer 2

Yes, task slots can be assigned to the same task manager given that each TM has 3 slots. If any node running active slot is down, the job will fail and will try to restart and at this point all the slots will be assigned on the only running node.