Not able to spin up Amazon EMR cluster with custom hive-site

0
votes

I am following the instructions in this tutorial to spin up a cluster.

http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-dev-create-metastore-outside.html

It is not working and it fails with an error "Bootstrap failure". But I don't see any errors in any logs too. It is making it extremely difficult for me to debug.

I have my mySQL database for metadata in RDS and it is active as well. Log folders do not have any errors. Do you know if this tutorial is complete?

1
amazon-web-serviceshiveamazon-emr
Hi, sorry you're running into this issue. I'm from the EMR team and can look into the root cause for you if you are able to provide the cluster ID of one of your clusters that failed. - Jonathan Kelly
j-3I7MISI0D159Z - passionate
Thanks. What I see is that the cluster timed out when provisioning because it was unable to connect to your external RDS instance. Perhaps your security group settings need to be modified (as mentioned in that page to which you linked)? - Jonathan Kelly
Am I missing the error message? I will change the security group now and try it. But it will be helpful if the error is shown as it will save lot of my time. - passionate
Unfortunately, you're not missing anything. Due to the way that the cluster is failing (timing out, actually), the only error message propagated back to you is that the cluster failed. Sorry about the inconvenience at this time, but there are ongoing projects for making certain types of failures more easily debuggable. - Jonathan Kelly

1 Answers

0
votes

It worked after providing RDS security group access to EMR security group and vice versa .