A connection error in remote mode of Titan-1.0.0+Hbase-0.98.20 using java

2
votes

I am learning Titan database. I have run it successfully in local-mode. Now, I am trying to use Titan database in "Remote Server Mode" introduced in Titan-documentation. My Titan version is Titan-1.0.0-hadoop1.

I have clusters in my LAN including cloud12 and cloud13. I installed hadoop-1.2.1 on it, the master is cloud12 and the slave is cloud13.

I want to test the performance about create a graph, so I design to start my Hbase-0.98.20 in pseudo-distributed-mode on machine cloud12 with independent zookeeper-3.4.6 and elasticsearch on cloud12.(I modified hbase-env.sh, and use default port 2181 in zoo.cfg)

Hadoop and HBase are seems like working regularly, I checked two servers by Jps and I also checked HBase through HBase shell.

Here are my configuration of hbase-site.xml:

<configuration>
<property>
<name>hbase.rootdir</name>
<value>hdfs://cloud12:9000/hbase</value>
</property>

<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>

<property>
<name>hbase.master.port</name>
<value>60000</value>
</property>

<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/home/Titan/hbase/zookeeperDir</value>
</property>

<property>
<name>hbase.tmp.dir</name>
<value>/home/Titan/hbase/tmpDir</value>
</property>

<property>
<name>hbase.zookeeper.property.clientPort</name>
<value>2181</value>
</property>

<property>
<name>zookeeper.znode.parent</name>
<value>/hbase-unsecure</value>
</property>

<property>
<name>hbase.zookeeper.property.maxClientCnxns</name>
<value>600</value>
</property>
</configuration>

Although when I ran my program in Eclipse on the other machine in LAN which named cloud6（I installed Titan-1.0.0 on this machine）, a confused error information printed in my output logs.

Here is the error constantly repeated in my output

789 [main] INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=192.168.12.148:2181 sessionTimeout=90000 watcher=hconnection-0x12d3a4e9, quorum=192.168.12.148:2181, baseZNode=/hbase

870 [main] INFO org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper - Process identifier=hconnection-0x12d3a4e9 connecting to ZooKeeper ensemble=192.168.12.148:2181

878 [main-SendThread(192.168.12.148:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server 192.168.12.148/192.168.12.148:2181. Will not attempt to authenticate using SASL (unknown error)

1030 [main-SendThread(192.168.12.148:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to 192.168.12.148/192.168.12.148:2181, initiating session

1049 [main-SendThread(192.168.12.148:2181)] INFO org.apache.zookeeper.ClientCnxn - Session establishment complete on server 192.168.12.148/192.168.12.148:2181, sessionid = 0x15654717951001b, negotiated timeout = 40000

1054 [main] INFO org.apache.hadoop.hbase.client.ZooKeeperRegistry - ClusterId read in ZooKeeper is null

ClusterId read in ZooKeeper is null？ I really confused about it. I tried to solve problem and I found my error are similar to many others, but I can't get clear answer from Google or other websites. Is there an error in my design of architecture? or configuration error?

I can ensure that my hosts and the time of cluster are all correct. Here is a part of my program to connect Hbase on cloud12 from cloud6, is anything wrong or lackness in my code ?

  public static final String INDEXNAME = "search"; 
  ...
  BaseConfiguration conf=new BaseConfiguration();
  conf.setProperty("storage.backend", "hbase");
  conf.setProperty("storage.hostname", "192.168.12.148");//ip of cloud12
  conf.setProperty("storage.tablename", "graph1");
  conf.setProperty("index." + INDEXNAME + ".backend", "elasticsearch");
  conf.setProperty("index." + INDEXNAME + ".hostname", "192.168.12.148"); 
  conf.setProperty("index." + INDEXNAME + ".elasticsearch.local-mode", false);  
  conf.setProperty("index." + INDEXNAME + ".elasticsearch.client-only", true);
2
javahadoophbasetitan
Note that the "ClusterId read in ZooKeeper is null" is an INFO level message. I don't think you mentioned it, but are you able to connect to graph1 with your code despite the INFO message? – Jason Plurad
@JasonPlurad Thanks for your response.I can't connect to graph1 with my code. On the one hand, I checked Hbase shell. On the other hand, I set some flags in my code by using java class ”Calendar“ to compute the time. It can be printed to .txt in local-mode. But in remote mode, above error informations repeated in output and finally seems like suspend. – Andrew Lee

2 Answers

2
votes

Some things to check since you are using an external Zookeeper ensemble:

  1. Add the hbase.zookeeper.quorum property value in hbase-site.xml. The value should be a comma-separated list of your Zookeeper nodes. It defaults to localhost.

  2. Add export HBASE_MANAGES_ZK=false to the hbase-env.sh. It defaults to true.

  3. Make sure that the zookeeper.znode.parent property value in hbase-site.xml matches the value in your Titan configuration with storage.hbase.ext.zookeeper.znode.parent. It defaults to /hbase. If these values don't match, the Titan connection to HBase will hang.

  4. Make sure that the clientPort property value in zoo.cfg matches the hbase.zookeeper.property.clientPort value in hbase-site.xml and also matches the storage.hbase.ext.hbase.zookeeper.property.clientPort value in your Titan configuration. It defaults to 2181. If these don't match, you would see connection exceptions in the logs.

  5. Make sure that the Zookeeper nodes are listening on the clientPort using an accessible IP address (not localhost).

Once you've verified that you can connect to your Titan table, I think that the INFO messages are safe to ignore. You can toggle the logging level through settings in log4j.properties:

log4j.logger.org.apache.zookeeper=WARN
log4j.logger.org.apache.hadoop.hbase.client=WARN
log4j.logger.org.apache.hadoop.hbase.zookeeper=WARN

There is also an open issue against Titan to investigate why the Zookeeper client connections are happening so frequently.

0
votes

i got this problem too. just upgrade zookeeper3.4.5 to zookeeper3.4.6,and synchronize the time, but there are many INFO message still, so i modified log4j like that:

log4j.properties: log4j.logger.org.apache.zookeeper=WARN log4j.logger.org.apache.hadoop.hbase.client=WARN log4j.logger.org.apache.hadoop.hbase.zookeeper=WARN

just ok,