org.apache.hadoop.hbase.client.NoServerForRegionException (ROOT region in transition)

Question

I'm trying to import some data from mysql into hbase by using sqoop. When the process running, i got the following error

13/07/23 12:38:58 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
13/07/23 12:38:58 INFO tool.CodeGenTool: Beginning code generation
13/07/23 12:38:59 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `pop` AS t LIMIT 1
13/07/23 12:38:59 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `pop` AS t LIMIT 1
13/07/23 12:38:59 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/local/hadoops
Note: /tmp/sqoop-hduser/compile/61f2e4f1d24e82a26240a5f1d4872baf/pop.java
uses or overrides a deprecated API. Note: Recompile with -Xlint:deprecation for details. 13/07/23 12:39:00 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-hduser/compile/61f2e4f1d24e82a26240a5f1d4872baf/pop.jar 13/07/23 12:39:00 WARN manager.MySQLManager: It looks like you are importing from mysql. 13/07/23 12:39:00 WARN manager.MySQLManager: This transfer can be faster! Use the --direct 13/07/23 12:39:00 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path. 13/07/23 12:39:00 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql) 13/07/23 12:39:00 INFO mapreduce.ImportJobBase: Beginning import of pop 13/07/23 12:39:00 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT 13/07/23 12:39:00 INFO zookeeper.ZooKeeper: Client environment:host.name=master 13/07/23 12:39:00 INFO zookeeper.ZooKeeper: Client environment:java.version=1.7.0_15 13/07/23 12:39:00 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation 13/07/23 12:39:00 INFO zookeeper.ZooKeeper: Client environment:java.home=/usr/lib/jvm/jdk1.7.0/jre 13/07/23 12:39:00 INFO zookeeper.ZooKeeper: Client environment:java.class.path=/usr/local/hadoops/libexec/../conf:/usr/lib/jvm/jdk1.7.0/lib/tools.jar:/usr/local/hadoops/libexec/..:/usr/local/hadoops/libexec/../hadoop-core-1.0.4.jar:/usr/local/hadoops/libexec/../lib/asm-3.2.jar:/usr/local/hadoops/libexec/../lib/aspectjrt-1.6.5.jar:/usr/local/hadoops/libexec/../lib/aspectjtools-1.6.5.jar:/usr/local/hadoops/libexec/../lib/commons-beanutils-1.7.0.jar:/usr/local/hadoops/libexec/../lib/commons-beanutils-core-1.8.0.jar:/usr/local/hadoops/libexec/../lib/commons-cli-1.2.jar:/usr/local/hadoops/libexec/../lib/commons-codec-1.4.jar:/usr/local/hadoops/libexec/../lib/commons-collections-3.2.1.jar:/usr/local/hadoops/libexec/../lib/commons-configuration-1.6.jar:/usr/local/hadoops/libexec/../lib/commons-daemon-1.0.1.jar:/usr/local/hadoops/libexec/../lib/commons-digester-1.8.jar:/usr/local/hadoops/libexec/../lib/commons-el-1.0.jar:/usr/local/hadoops/libexec/../lib/commons-httpclient-3.0.1.jar:/usr/local/hadoops/libexec/../lib/commons-io-2.1.jar:/usr/local/hadoops/libexec/../lib/commons-lang-2.4.jar:/usr/local/hadoops/libexec/../lib/commons-logging-1.1.1.jar:/usr/local/hadoops/libexec/../lib/commons-logging-api-1.0.4.jar:/usr/local/hadoops/libexec/../lib/commons-math-2.1.jar:/usr/local/hadoops/libexec/../lib/commons-net-1.4.1.jar:/usr/local/hadoops/libexec/../lib/core-3.1.1.jar:/usr/local/hadoops/libexec/../lib/hadoop-capacity-scheduler-1.0.4.jar:/usr/local/hadoops/libexec/../lib/hadoop-fairscheduler-1.0.4.jar:/usr/local/hadoops/libexec/../lib/hadoop-thriftfs-1.0.4.jar:/usr/local/hadoops/libexec/../lib/hsqldb-1.8.0.10.jar:/usr/local/hadoops/libexec/../lib/jackson-core-asl-1.8.8.jar:/usr/local/hadoops/libexec/../lib/jackson-mapper-asl-1.8.8.jar:/usr/local/hadoops/libexec/../lib/jasper-compiler-5.5.12.jar:/usr/local/hadoops/libexec/../lib/jasper-runtime-5.5.12.jar:/usr/local/hadoops/libexec/../lib/jdeb-0.8.jar:/usr/local/hadoops/libexec/../lib/jersey-core-1.8.jar:/usr/local/hadoops/libexec/../lib/jersey-json-1.8.jar:/usr/local/hadoops/libexec/../lib/jersey-server-1.8.jar:/usr/local/hadoops/libexec/../lib/jets3t-0.6.1.jar:/usr/local/hadoops/libexec/../lib/jetty-6.1.26.jar:/usr/local/hadoops/libexec/../lib/jetty-util-6.1.26.jar:/usr/local/hadoops/libexec/../lib/jsch-0.1.42.jar:/usr/local/hadoops/libexec/../lib/junit-4.5.jar:/usr/local/hadoops/libexec/../lib/kfs-0.2.2.jar:/usr/local/hadoops/libexec/../lib/log4j-1.2.15.jar:/usr/local/hadoops/libexec/../lib/mockito-all-1.8.5.jar:/usr/local/hadoops/libexec/../lib/oro-2.0.8.jar:/usr/local/hadoops/libexec/../lib/servlet-api-2.5-20081211.jar:/usr/local/hadoops/libexec/../lib/slf4j-api-1.4.3.jar:/usr/local/hadoops/libexec/../lib/slf4j-log4j12-1.4.3.jar:/usr/local/hadoops/libexec/../lib/xmlenc-0.52.jar:/usr/local/hadoops/libexec/../lib/jsp-2.1/jsp-2.1.jar:/usr/local/hadoops/libexec/../lib/jsp-2.1/jsp-api-2.1.jar:/usr/local/sqoops/bin/../conf::/usr/local/sqoops/bin/../lib/activation-1.1.jar:/usr/local/sqoops/bin/../lib/ant-contrib-1.0b3.jar:/usr/local/sqoops/bin/../lib/ant-eclipse-1.0-jvm1.2.jar:/usr/local/sqoops/bin/../lib/asm-3.1.jar:/usr/local/sqoops/bin/../lib/avro-1.5.3.jar:/usr/local/sqoops/bin/../lib/avro-ipc-1.5.3.jar:/usr/local/sqoops/bin/../lib/avro-mapred-1.5.3.jar:/usr/local/sqoops/bin/../lib/commons-beanutils-1.7.0.jar:/usr/local/sqoops/bin/../lib/commons-beanutils-core-1.8.0.jar:/usr/local/sqoops/bin/../lib/commons-cli-1.2.jar:/usr/local/sqoops/bin/../lib/commons-codec-1.4.jar:/usr/local/sqoops/bin/../lib/commons-collections-3.2.1.jar:/usr/local/sqoops/bin/../lib/commons-configuration-1.6.jar:/usr/local/sqoops/bin/../lib/commons-digester-1.8.jar:/usr/local/sqoops/bin/../lib/commons-el-1.0.jar:/usr/local/sqoops/bin/../lib/commons-httpclient-3.1.jar:/usr/local/sqoops/bin/../lib/commons-io-1.4.jar:/usr/local/sqoops/bin/../lib/commons-io-2.1.jar:/usr/local/sqoops/bin/../lib/commons-lang-2.5.jar:/usr/local/sqoops/bin/../lib/commons-logging-1.1.1.jar:/usr/local/sqoops/bin/../lib/commons-math-2.1.jar:/usr/local/sqoops/bin/../lib/commons-net-1.4.1.jar:/usr/local/sqoops/bin/../lib/core-3.1.1.jar:/usr/local/sqoops/bin/../lib/guava-11.0.2.jar:/usr/local/sqoops/bin/../lib/hadoop-core-1.0.4.jar:/usr/local/sqoops/bin/../lib/hbase-0.94.8.jar:/usr/local/sqoops/bin/../lib/high-scale-lib-1.1.1.jar:/usr/local/sqoops/bin/../lib/hsqldb-1.8.0.10.jar:/usr/local/sqoops/bin/../lib/httpclient-4.1.2.jar:/usr/local/sqoops/bin/../lib/httpcore-4.1.3.jar:/usr/local/sqoops/bin/../lib/jackson-core-asl-1.7.3.jar:/usr/local/sqoops/bin/../lib/jackson-core-asl-1.8.8.jar:/usr/local/sqoops/bin/../lib/jackson-jaxrs-1.8.8.jar:/usr/local/sqoops/bin/../lib/jackson-mapper-asl-1.7.3.jar:/usr/local/sqoops/bin/../lib/jackson-mapper-asl-1.8.8.jar:/usr/local/sqoops/bin/../lib/jackson-xc-1.8.8.jar:/usr/local/sqoops/bin/../lib/jamon-runtime-2.3.1.jar:/usr/local/sqoops/bin/../lib/jasper-compiler-5.5.23.jar:/usr/local/sqoops/bin/../lib/jasper-runtime-5.5.23.jar:/usr/local/sqoops/bin/../lib/jaxb-api-2.1.jar:/usr/local/sqoops/bin/../lib/jaxb-impl-2.2.3-1.jar:/usr/local/sqoops/bin/../lib/jersey-core-1.8.jar:/usr/local/sqoops/bin/../lib/jersey-json-1.8.jar:/usr/local/sqoops/bin/../lib/jersey-server-1.8.jar:/usr/local/sqoops/bin/../lib/jettison-1.1.jar:/usr/local/sqoops/bin/../lib/jetty-6.1.26.jar:/usr/local/sqoops/bin/../lib/jetty-util-6.1.26.jar:/usr/local/sqoops/bin/../lib/jopt-simple-3.2.jar:/usr/local/sqoops/bin/../lib/jruby-complete-1.6.5.jar:/usr/local/sqoops/bin/../lib/jsp-2.1-6.1.14.jar:/usr/local/sqoops/bin/../lib/jsp-api-2.1-6.1.14.jar:/usr/local/sqoops/bin/../lib/jsr305-1.3.9.jar:/usr/local/sqoops/bin/../lib/junit-4.10-HBASE-1.jar:/usr/local/sqoops/bin/../lib/libthrift-0.8.0.jar:/usr/local/sqoops/bin/../lib/log4j-1.2.16.jar:/usr/local/sqoops/bin/../lib/metrics-core-2.1.2.jar:/usr/local/sqoops/bin/../lib/mysql-connector-java-5.1.25-bin.jar:/usr/local/sqoops/bin/../lib/netty-3.2.4.Final.jar:/usr/local/sqoops/bin/../lib/paranamer-2.3.jar:/usr/local/sqoops/bin/../lib/protobuf-java-2.4.0a.jar:/usr/local/sqoops/bin/../lib/servlet-api-2.5-6.1.14.jar:/usr/local/sqoops/bin/../lib/slf4j-api-1.4.3.jar:/usr/local/sqoops/bin/../lib/slf4j-log4j12-1.4.3.jar:/usr/local/sqoops/bin/../lib/snappy-java-1.0.3.2.jar:/usr/local/sqoops/bin/../lib/stax-api-1.0.1.jar:/usr/local/sqoops/bin/../lib/velocity-1.7.jar:/usr/local/sqoops/bin/../lib/xmlenc-0.52.jar:/usr/local/sqoops/bin/../lib/zookeeper-3.4.5.jar:/usr/local/sqoops/bin/../sqoop-1.4.3.jar:/usr/local/sqoops/bin/../sqoop-test-1.4.3.jar: 13/07/23 12:39:00 INFO zookeeper.ZooKeeper: Client environment:java.library.path=/usr/local/hadoops/libexec/../lib/native/Linux-amd64-64 13/07/23 12:39:00 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp 13/07/23 12:39:00 INFO zookeeper.ZooKeeper: Client environment:java.compiler= 13/07/23 12:39:00 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux 13/07/23 12:39:00 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64 13/07/23 12:39:00 INFO zookeeper.ZooKeeper: Client environment:os.version=3.5.0-23-generic 13/07/23 12:39:00 INFO zookeeper.ZooKeeper: Client environment:user.name=hduser 13/07/23 12:39:00 INFO zookeeper.ZooKeeper: Client environment:user.home=/home/hduser 13/07/23 12:39:00 INFO zookeeper.ZooKeeper: Client environment:user.dir=/home/hduser 13/07/23 12:39:00 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=180000 watcher=hconnection 13/07/23 12:39:00 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 6256@Master 13/07/23 12:39:00 INFO zookeeper.ClientCnxn: Opening socket connection to server ip6-localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) 13/07/23 12:39:00 INFO zookeeper.ClientCnxn: Socket connection established to ip6-localhost/127.0.0.1:2181, initiating session 13/07/23 12:39:00 INFO zookeeper.ClientCnxn: Session establishment complete on server ip6-localhost/127.0.0.1:2181, sessionid = 0x14009b9169f0009, negotiated timeout = 180000 13/07/23 12:39:00 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=180000 watcher=catalogtracker-on-org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@3b48a380 13/07/23 12:39:00 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 6256@Master 13/07/23 12:39:00 INFO zookeeper.ClientCnxn: Opening socket connection to server ip6-localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) 13/07/23 12:39:00 INFO zookeeper.ClientCnxn: Socket connection established to ip6-localhost/127.0.0.1:2181, initiating session 13/07/23 12:39:00 INFO zookeeper.ClientCnxn: Session establishment complete on server ip6-localhost/127.0.0.1:2181, sessionid = 0x14009b9169f000a, negotiated timeout = 180000 13/07/23 12:49:01 INFO zookeeper.ZooKeeper: Session: 0x14009b9169f000a closed 13/07/23 12:49:01 INFO zookeeper.ClientCnxn: EventThread shut down 13/07/23 12:49:01 ERROR tool.ImportTool: Encountered IOException running import job: org.apache.hadoop.hbase.client.NoServerForRegionException: Unable to find region for after 10 tries. at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:991) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:896) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:857) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:233) at org.apache.hadoop.hbase.client.HTable.(HTable.java:173) at org.apache.hadoop.hbase.catalog.MetaReader.getHTable(MetaReader.java:201) at org.apache.hadoop.hbase.catalog.MetaReader.getMetaHTable(MetaReader.java:227) at org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:722) at org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:184) at org.apache.hadoop.hbase.catalog.MetaReader.tableExists(MetaReader.java:450) at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:250) at org.apache.sqoop.mapreduce.HBaseImportJob.jobSetup(HBaseImportJob.java:194) at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:219) at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:545) at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:111) at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:403) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476) at org.apache.sqoop.Sqoop.run(Sqoop.java:145) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229) at org.apache.sqoop.Sqoop.main(Sqoop.java:238)

In the 'regions in transition', i got the following information: -ROOT-,,0.70236052 state=OFFLINE, ts=Tue Jul 23 12:42:08 WIT 2013 (594s ago), server=null

Can anyone please help me to solve this? I'm out of ideas. I've got a presentation one week later. Thank you for the kindness to help me.

Tariq Tariq · Accepted Answer · 2013-07-23T20:24:30

Looks like some problem with your RegionServer. Perhaps, RegionServer's session has expired or it is dead and hence the HMaster is not able to assign the regions held by this RegionServer. Looking at the RegionServer logs would be helpful. You could probably see in HMaster logs where this region was assigned. Then go to that RegionServer's UI and see if you see the region?

BTW, How many nodes do you have?Is the data too huge to be stored in your current cluster?

org.apache.hadoop.hbase.client.NoServerForRegionException (ROOT region in transition)

1 Answers