HBase fails to start in single node cluster mode on Mac OSX

2
votes

I am trying to get a personal HBase development environment set up. I have hdfs and yarn running, but cannot get HBase to start.

I have started up hadoop 2.7.1, by running start-dfs.sh and start-yarn.sh. I have verified these are running by testing hdfs dfs -mkdir /test and running a sample MR job bundled in the examples, I have browsed HDFS at port 50070.

I have started zookeeper 3.4.6 on port 2181 and set its dataDir. My zoo.cfg has:

dataDir=/Users/.../tools/hd/zookeeper_data
clientPort=2181

I observe its zookeeper_server.PID file in the dataDir I chose, and when I run jps I see the below:

51074 NodeManager
50743 DataNode
50983 ResourceManager
50856 SecondaryNameNode
57848 QuorumPeerMain
58731 Jps
50653 NameNode

QuorumPeerMain above matches the PID in zookeeper_server.PID, as I would expect. Is this expectation correct? From what I have done so far, should it be expected that any more processes should be showing here?

I installed hbase-1.1.2. I configure hbase-site.xml. I set the hbase.rootDir to be hdfs://localhost:8200/hbase, my hdfs is running at localhost:8200. I set hbase.zookeeper.property.dataDir to my zookeeper's dataDir, with the expectation that it will use this property to find the PID of a running zookeeper. Is this expectation correct or have I misunderstood? The config in hbase-site.xml is:

<property>
<name>hbase.rootdir</name>
<value>hdfs://localhost:8020/hbase</value>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>Users/.../tools/hd/zookeeper_data</value>
</property>

When I run start-hbase.sh my server fails to start. I see this log message:

2015-09-26 19:32:43,617 ERROR [main] master.HMasterCommandLine: Master exiting

To investigate I ran hbase master start and get more detail:

2015-09-26 19:41:26,403 INFO [Thread-1] server.NIOServerCnxn: Stat command output 2015-09-26 19:41:26,405 INFO [Thread-1] server.NIOServerCnxn: Closed socket connection for client /127.0.0.1:63334 (no session established for client) 2015-09-26 19:41:26,406 INFO [main] zookeeper.MiniZooKeeperCluster: Started MiniZooKeeperCluster and ran successful 'stat' on client port=2182 Could not start ZK at requested port of 2181. ZK was started at port: 2182. Aborting as clients (e.g. shell) will not be able to find this ZK quorum. 2015-09-26 19:41:26,406 ERROR [main] master.HMasterCommandLine: Master exiting java.io.IOException: Could not start ZK at requested port of 2181. ZK was started at port: 2182. Aborting as clients (e.g. shell) will not be able to find this ZK quorum. at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:214) at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:139) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126) at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2304)

So I have a few questions:

  1. Should I be trying to set up a zookeeper before running HBase?
  2. Why when I have started a zookeeper and told HBase where its dataDir is, does HBase try to start its own zookeeper?
  3. Anything obviously stupid/misguided in the above?
2
hadoophbaseapache-zookeeper

2 Answers

1
votes

The script you are using to start hbase start-hbase.sh will try to start the following components, in order:

  • zookeeper
  • hbase master
  • hbase regionserver
  • hbase master-backup

So, you could either stop the zookeeper which is started by you (or) you could start the daemons individually yourself:

# start hbase master
bin/hbase-daemon.sh --config ${HBASE_CONF_DIR} start master
# start region server
bin/hbase-daemons.sh --config ${HBASE_CONF_DIR} --hosts ${HBASE_CONF_DIR}/regionservers start regionserver
0
votes

HBase stand alone starts it's own zookeeper (if you run start-hbase.sh), but it if fails to start or keep running, the other need hbase daemons won't work.

Make sure you explicitly set the properties for your interface lo0 in the hbase-site.xml file:

<property>
  <name>hbase.zookeeper.dns.interface</name>
  <value>lo0</value>
</property>

<property>
  <name>hbase.regionserver.dns.interface</name>
  <value>lo0</value>
</property>

<property>
  <name>hbase.master.dns.interface</name>
  <value>lo0</value>
</property>

I found that when my wifi was on, if these entries were missing, zookeeper filed to start.