EMR activity stuck in Waiting_For_Runner state

8
votes

I am creating a data pipeline to export dynamoDB table to S3 bucket.I used the standard template to use for this in data pipeline console. I ha verified that the runsOn field is set to the name of EMR cluster to be started. However, The EMR activity status is still as Waiting_For_Runner...Any ideas why is this so??

Thanks!!!

1
amazon-web-servicesemramazon-data-pipeline
solved the issue.. authorization problem :( – user3610975
How did you fix this authorization issue? – Cognitio

1 Answers

1
votes

Waiting_For_Runner means datatpipeline is trying to connect to EMR.

Few reasons you can check:

  1. IAM permissions between EMR and Data pipeline(Roles). Here's a link!
  2. Check Task runner is running or not on master instance
    $ps -ef | grep workerGroup (Master Instance)
  3. Check --workgroup name in EMR(you can see in Task runner process) and compare it with the name of workgroup in datapipeline.