Ganglia and Amazon Elastic Map Reduce - install issues

Question

Following the instructions for "Initializing Ganglia on a Job Flow" I get my cluster up but don't see any Ganglia process running (on 8157).

http://docs.amazonwebservices.com/ElasticMapReduce/latest/DeveloperGuide/init_Ganglia.html

elastic-mapreduce --create --alive --name "Tom's Daily Hive 8x Flow" --instance-type c1.medium --num-instances 8 --availability-zone us-east-1a --bootstrap-action "s3://elasticmapreduce/bootstrap-actions/install-ganglia" --stream

There is a 0K file in: /tmp/ganglia-installed

Any suggestions? Thanks!

Answer 1

OK ... so it turns out the instructions at Amazon are a bit light on the details.

Try here http://ecs-network.serv.pacific.edu/ecpe-293a/projects/commoncrawl-tutorial and here http://docs.amazonwebservices.com/ElasticMapReduce/latest/DeveloperGuide/UsingtheHadoopUserInterface.html#AccessingtheHadoopUserInterfacetoMonitorJobStatus2

and you will do much better.

I am light on port forwarding experience, and what they have in the EMR instructions led me down many wrong paths (guesses).

Answer 2

Your create cluster command looks OK. Set up FoxyProxy on Firefox like they mention in the same document you are referring to in your question.

After the set up, go to localhost:8157/ganglia, that should take you to your monitoring graphs!