Hive unable to load data- cloudera quickstart VM 5.8

1
votes

I am using cloudera quickstart VM 5.8 as hadoop environment. I tried to do the following.

  1. Created a hive table using hdfs absolute uri

CREATE EXTERNAL TABLE IF NOT EXISTS stocks_tb ( exch STRING, symbol STRING, ymd STRING, price_open FLOAT, price_high FLOAT, price_low FLOAT, price_close FLOAT, volume INT, price_adj_close FLOAT) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LOCATION 'hdfs://quickstart.cloudera:8020/hadoop/hive/stocks';

{I tried using relative path but hive CLI complained me that I am using relative path where absolute path is required}

  1. Loaded the dataset stocks from local filesystem to input/stocks in HDFS.

  2. Tried to load the dataset into table

LOAD DATA INPATH 'hdfs://quickstart.cloudera:8020/input/stocks' INTO TABLE stocks_tb;

I get the error

Invalid path ''hdfs://quickstart.cloudera:8020/input/stocks'': 
No files matching path hdfs://quickstart.cloudera:8020/input/stocks

I don't know why I am getting this error. The data exist in input/stocks/stocks but the error occurs.

could you spot the error/ any idea about the error?

Also when I do

hadoop fs -ls hdfs://quickstart.cloudera:8020/input/stocks

I am not able to see any file but when I do

hadoop fs -ls input/stocks

I see the stocks file

I don't know how to interpret this. Could you help?

Thanks

2
hadoophiveclouderacloudera-quickstart-vm

2 Answers

1
votes

Just as what I suspected.

The data was actually in user/cloudera/input/stocks and I was referring to /input/stocks in the load command.

The correct load command would be

LOAD DATA INPATH 'hdfs://quickstart.cloudera:8020/user/cloudera/input/stocks' INTO TABLE stocks_tb;

0
votes

It is likely your user may not have permission to read/write to /input/stocks in HDFS.

Which user are you running the program as? You could try using the hdfs superuser account.

What is the ownership of /input/stocks?

Run the following to check the ownership

hadoop fs -ls /
hadoop fs -ls /input

You can run the following to set ownership/permissions;

hadoop fs -chown user:group /input