AWS EMR YARN capacity scheduler configuration

1
votes

I am trying to start an EMR cluster with bootstrap actions to configure YARN scheduler. This is the article I used to find the values.

http://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-object-hadoopactivity.html http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/emr-plan-bootstrap.html#PredefinedBootstrapActions_ConfigureHadoop

Below is the bootstrap action:

s3://us-east-1.elasticmapreduce/bootstrap-actions/configure-hadoop -z, yarn.scheduler.capacity.root.queues=low\,high\,default, -z, yarn.scheduler.capacity.root.high.capacity=50, -z, yarn.scheduler.capacity.root.low.capacity=10, -z, yarn.scheduler.capacity.root.default.capacity=30

AMI version: 3.9.0 Hadoop distribution:Amazon 2.4.0

But I always get this error On the master instance (i-72ba3ca7), after bootstrap actions were run Hadoop failed to launch

What am I missing?

1
javaamazon-web-servicesemr

1 Answers

1
votes

One thing, I could see is, the sum of below should be 100 which is 90(50+10+30)

yarn.scheduler.capacity.root.high.capacity=50, -z, yarn.scheduler.capacity.root.low.capacity=10, -z, yarn.scheduler.capacity.root.default.capacity=30

Please modify the above numbers to make it 100 and try it.