How to run Flink Job Manager and Task Manager on different nodes?

1
votes

I have a long-running Flink cluster on Yarn. This is the command to start it sudo -E flink run -m yarn-cluster -ynm FlinkStreamBootstrap -yn 2 -s ${SAVEPOINT} -d -c flink.FlinkBootstrap ${FLINK_LIB}/flink_2.11-${JAR_VERSION}-assembly.jar --jobName "Flink Job"

Sometimes the task manager is running on the same node as the job manager. Sometimes is different. Is there a way to specify to run them on different nodes to start with?

Thanks

1
apache-flink

1 Answers

2
votes

Flink currently does not support expressing placement constraints for Yarn containers. Flink has been developed to be location agnostic and it is the responsibility of the underlying cluster manager to decide where to allocate resources. This property has the benefit that it does not matter whether the Yarn cluster has only a single very beefy node where it can place multiple containers onto or multiple smaller nodes.