How to run flink jar file in Mesos cluster

1
votes

I have two nodes that docker have been installed on them. Moreover, I have installed Apache Mesos on docker. First, I run this command in docker which does not show anything in result:

 /home/flink-1.7.0/bin/mesos-appmaster.sh -Djobmanager.heap.mb=1024 -Djobmanager.rpc.port=6123 -Drest.port=8081 -Dmesos.resourcemanager.tasks.mem=1024 -Dtaskmanager.heap.mb=1024 -Dtaskmanager.numberOfTaskSlots=2 -Dparallelism.default=2 -Dmesos.resourcemanager.tasks.cpus=1

After that I run this command to run my jar file:

/home/flink-1.7.0/bin/flink run -m 172.28.10.133:5050 /home/Flink_Cassandra_test_jar/Flink_Cassandra_test.jar

After that, I get this error:

org.apache.flink.client.deployment.ClusterRetrieveException: Couldn't retrieve standalone cluster

By the way, I configured Zookeeper too. Also, I added this two line in Flink yaml file:

 jobmanager.rpc.address: 150.20.11.133
 high-availability: zookeeper
 high-availability.zookeeper.quorum: 150.20.11.133:2181

Would you please tell me how to run flink jar file on mesos cluster?

Any help would be appreciated.

1
dockerapache-flinkapache-zookeepermesos

1 Answers

1
votes

Fill this parameters on the flink-conf.yaml file.

mesos.master: IP-MASTER-MESOS:5050
mesos.initial-tasks: 10
mesos.resourcemanager.tasks.container.type: mesos
jobmanager.heap.mb: 1024
jobmanager.web.address: 130.239.48.136
jobmanager.web.port: 8081
mesos.resourcemanager.tasks.mem: 4096
taskmanager.heap.mb: 3500
mesos.resourcemanager.tasks.cpus: 2
mesos.resourcemanager.tasks.disk: 4096
mesos.resourcemanager.tasks.taskmanager-cmd: "/home/flink/flink-1.9.0/bin/mesos-taskmanager.sh"
mesos.resourcemanager.artifactserver.ssl.enabled: false
mesos.resourcemanager.framework.name: "FLINK_on_MESOS_intensive_cpu_usage"

then run your application:

/home/flink/flink-1.9.0/bin/flink run -c org.sense.flink.App /home/flink/explore-flink/target/you-application-flink.jar

Mesos will dispose task manager, slots in demand to Flink.