How to use dolphinscheduler with AWS EMR

0
votes

We have installed Dolphinscheduler 1.3 on a cluster of 2 ec2 instances. We would like to run a scheduled flink job.

We have an AWS EMR cluster and we have Flink running on it.

How can I have DS manage jobs on the EMR cluster? The only worker group I see on DS is the "default" worker group, which contains the 2 instances we installed DS on.

1
apache-flinkamazon-emr

1 Answers

0
votes

you can config the worker groups in the conf/worker.properties,

worker.groups=default,wokergroup1,workergroup2,xxx,...

then restart the worker server process

if your configuration works, you will see the all worker group names on the UI: image about all work group names