I have few long running (streaming) flink jobs which I want to run on AWS EMR. I read the steps to do so from AWS console or using aws cli. But this is something manual. What are the ways to automate this in a production environment (where I can have multiple jobs and may need to handle failure scenarios as well)?
1 Answers
1
votes
If you want to submit multiple jobs to an EMR cluster, you could use Flink's REST API to submit and monitor jobs. It uses the same port as the web UI, which you can access on EMR by following these instructions.
If you want to spin up a new EMR cluster for each Flink job, you can use AWS's API or CLI.
