Performance monitoring for Google Cloud DataProc

Question

We are using GoogleCloudPlatform for big-data analytics. For processing we are currently using the google cloud dataproc & spark-streaming.

We would like to check the job performance using some monitoring utilities like Ganglia, Graphite, Dr.Elephant etc.

For setting up of any of these requires modification of configuration in each nodes. I would like to know, if the dataproc natively supports any of such performance monitoring tools?

Answer 1

None of those are supported out of the box, but you could add Ganglia using this initialization action. You can configure it however you like by editing the script.

You would then have to follow these instructions to access the Web UI.