How many slots needs to provide in the flink, If all the server configurations are different?

0
votes

I am running flink on the kubernetes.

Following are the configurations of server:

server - 1: 4cpu, 16gb RAM
server - 2: 2cpu, 8gb RAM
server - 3: 2cpu, 8gb RAM

By using the above configurations how to set the parallelism and slots.  I want to run 3 taskmanger. Currently I have provided below configurations. Is it correct?

taskmanager.numberOfTaskSlots: 2
parallelism.default: 3

As per the understanding each cpu will be treated as 1 slot. should it be like numberOfTaskSlots has to be configure 2, because that is available for 3 taskmanager.

parallelism.default: 3 because i am using number of taskmanager as 3.

1
apache-flink

1 Answers

2
votes

The sum of the parallelism across all of the jobs should match the total number of slots. The slots should each have the same resources available (RAM and cpus). One cpu per slot is just a guideline -- what's appropriate really depends on the job(s) that will be running, and how much work each slot will be doing.

In this case with 2 slots per task manager, and 3 task managers, the cluster can support a total parallelism of 6.