Apache Flink: number of TaskManagers per machine

Question

The number of CPU cores per machine is four. In flink standalone mode, how should I set the number of TaskManagers on each machine?

1. 1 TaskManager, each TaskManager has 4 slots.
2. 2 TaskManagers, each TaskManager has 2 slots.
3. 4 TaskManagers, each TaskManager has 1 slot. This setting is like apache-storm.

Answer 1

Normally you'd have one TaskManager per server, and (as per the doc that bupt_ljy referenced) one slot per physical CPU core. So I'd go with your option #1.

Answer 2

There's also the consideration of Flink's scheduling algorithm. We've frequently run into problems where, with multiple hosts running one large task manager a piece, all jobs get scheduled to one host, which can cause load problems.

We ended up making multiple smaller task managers per host and jobs seem to be distributed better (although they still cluster on one node often).

So, in my experience, I'd lean more towards 4 task managers with 1 slot a piece, or maybe compromise at 2 task managers with 2 slots a piece.

Answer 3

I think it depends on your application.

In official documents Distributed Runtime Environment, it says As a rule-of-thumb, a good default number of task slots would be the number of CPU cores. With hyper-threading, each slot then takes 2 or more hardware thread contexts.

But if you have to use a lot of memory in your application, then you don't need too many slots in one task manager.