Loading data from HDFS to HBASE

Question

I'm using Apache hadoop 1.1.1 and Apache hbase 0.94.3.I wanted to load data from HDFS to HBASE. I wrote pig script to serve the purpose. First i created hbase table in habse and next wrote pig script to load the data from HDFS to HBASE. But it is not loading the data into hbase table. Dont know where it's going worng. Below is the command used to craete hbase table :

create table 'mydata','mycf'

Below is the pig script to load data from hdfs to hbase:

    A = LOAD '/user/hduser/Dataparse/goodrec1.txt' USING PigStorage(',') as (c1:int, c2:chararray,c3:chararray,c4:int,c5:chararray);   
    STORE A INTO 'hbase://mydata'
    USING org.apache.pig.backend.hadoop.hbase.HBaseStorage(
          'mycf:c1,mycf:c2,mycf:c3,mycf:c4,mycf:c5');

After excecuting the script it says

2014-04-29 16:01:06,367 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete

2014-04-29 16:01:06,376 [main] ERROR org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
2014-04-29 16:01:06,382 [main] INFO  org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:

HadoopVersion   PigVersion      UserId  StartedAt       FinishedAt      Features
1.1.1   0.12.0  hduser  2014-04-29 15:58:07     2014-04-29 16:01:06     UNKNOWN

Failed!

Failed Jobs:
JobId   Alias   Feature Message Outputs
job_201403142119_0084   A       MAP_ONLY        Message: Job failed! Error -           JobCleanup Task Failure, Task: task_201403142119_0084_m_000001     hbase://mydata,

Input(s):
Failed to read data from "/user/hduser/Dataparse/goodrec1.txt"

Output(s):
Failed to produce result in "hbase://mydata"

Counters:
Total records written : 0
Total bytes written : 0
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0

Job DAG:
job_201403142119_0084


2014-04-29 16:01:06,382 [main] INFO   org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed!

Please help where i'm going wrong ?

What does hadoop fs -ls /user/hduser/Dataparse/file.txt show?? — Tariq
file.txt is text file with comma separated values and its content is: 15,,CO,5656,F 20,George11,MI,5555,M 1,8888a,FL,5899,F — shree11
I was not asking about for the content of the file. I just wanted the output of hadoop fs -ls /user/hduser/Dataparse/goodrec1.txt..Please copy the exact output along with the command. — Tariq
hadoop fs -cat /user/hduser/Dataparse/goodrec1.txt 15,,CO,5656,F 20,George11,MI,5555,M 1,8888a,FL,5899,F — shree11
can you provide result of list command ran in hbase shell? — psmith

bridiver bridiver · Accepted Answer · 2014-04-29T12:41:10

You have specified too many columns in the output to hbase. You have 5 input columns and 5 output columns, but HBaseStorage requires the first column to be the row key so there should only be 4 in the output

Loading data from HDFS to HBASE

1 Answers