What if the number of task slot is greater than the task manager vcores

Question

I have set yarn.container.vcores=1 in the yaml configuration file, and I kick off a flink cluster with

yarn-session.sh -jm 1024 -tm 2048 -n 4 -s 3

With above command, I kick off 4 Task Managers, and each TM is allocated only 1 vcore, also each TM has 3 task slots.

Assume if I have 3 subtasks run in the TM, every task slot owns one sub task.

Could these 3 sub task run simultaneously with 3 threads(using 3 cpu cores, vcore=1 doesn't take effect) or they will concurrently with only one thread(vcore = 1 takes effect)

Answer 1

Every task will have its own JVM thread, regardless of how many cores are available. It's fine if a task manager has 3 slots and 1 vcore. The tasks running in those slots will simply contend for the resources available to the TM.

Note that normally you will have several subtasks (or tasks -- tasks and subtasks are really the same thing) running in one slot. For each task slot to have only one subtask, either the job is super simple (i.e., a job graph with only one node), or you've done something special to isolate the subtasks.