H2O: unable to connect to h2o cluster through python

Question

I have a 5 node hadoop cluster running HDP 2.3.0. I setup a H2O cluster on Yarn as described here.

On running following command

hadoop jar h2odriver_hdp2.2.jar water.hadoop.h2odriver -libjars ../h2o.jar -mapperXmx 512m -nodes 3 -output /user/hdfs/H2OTestClusterOutput

I get the following ouput

H2O cluster (3 nodes) is up
(Note: Use the -disown option to exit the driver after cluster formation)
(Press Ctrl-C to kill the cluster)
Blocking until the H2O cluster shuts down...

When I try to execute the command

h2o.init(ip="10.113.57.98", port=54321)

The process remains stuck at this stage.On trying to connect to the web UI using the ip:54321, the browser tries to endlessly load the H2O admin page but nothing ever displays.

On forcefully terminating the init process I get the following error

No instance found at ip and port: 10.113.57.98:54321. Trying to start local jar...

However if I try and use H2O with python without setting up a H2O cluster, everything runs fine.

I executed all commands as the root user. Root user has permissions to read and write from the /user/hdfs hdfs directory.

I'm not sure if this is a permissions error or that the port is not accessible.

Any help would be greatly appreciated.

Have you tried using the -network option when running hadoop jar? There maybe multiple network interfaces on your cluster and H2O may not be binding to the correct one. Do a "ifconfig" on the terminal prompt to see what network interfaces there are. Also by looking at the H2O log file while it is starting, you'll see which ip address it binds to. The -network option will let you change the default address. — Mark Chan

Nick Karpov Nick Karpov · Accepted Answer · 2016-03-10T00:34:14

It looks like you are using H2O2 (H2O Classic). I recommend upgrading your H2O to the latest (H2O 3). There is a build specifically for HDP2.3 here: http://www.h2o.ai/download/h2o/hadoop

Running H2O3 is a little cleaner too:

hadoop jar h2odriver.jar -nodes 1 -mapperXmx 6g -output hdfsOutputDirName

Also, 512mb per node is tiny - what is your use case? I would give the nodes some more memory.

H2O: unable to connect to h2o cluster through python

1 Answers