1
votes

I have a Flink cluster with 8 CPU cores and 16GB RAM. When I submit a job to the cluster using the command: flink run -m ip:port -p 8 X.jar

in job manager web interface I can see all the 8 cores are used. According to the documents, Flink standalone cluster manager runs applications using FIFO scheduling. I want to know Does Flink standalone cluster manager support running application simultaneously on the cluster, something like round-robin? Or I should use YARN?

1

1 Answers

1
votes

As long as you have enough task slots on the task managers your jobs will be executed simultaneously by the Flink. Not in the round-robin fashion, but together at once in separate threads. Parallelism is the parameter that let you to control the number of those threads used by a single Task:

https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/parallel.html

However for various reasons, mostly related to resource management (it is impossible/difficult to control/limit/assign/track the resources like memory of two different logical applications running inside a single JVM), it is recommended to start one Flink cluster per application. YARN is one way to achieve that (documented here). It will allow you to control the resources on a per container basis.