External Hive Table with Partition not reading the file

Question

I have created the external hive table like below and tried to read the file in the location.

sample input:

c1,c2,c3,c4,c5
ass,adda,ada,er,asa
asdasd,asd,asas,qwqw,dfdf

Extended table with partition

create external table tablename(field3 varchar(50), field4 varchar(50), filed5 varchar(50))
partitioned by (field1 varchar(50), field2 varchar(50))
ROW FORMAT DELIMITED
fields terminated by ','
lines terminated by '\n'
location '/path/to/Folder/'
tblproperties ("skip.header.line.count"="1");

Only one csv file in the folder location which has 5 columns and 1k rows

After creating the table when I tried to run select query, no results is showing.

Then I tried creating the external table without partitioning like below and getting output when I run select query.

create external table tablename(field1 varchar(50), field2 varchar(50),field3 varchar(50), field4 varchar(50), filed5 varchar(50))
    ROW FORMAT DELIMITED
    fields terminated by ','
    lines terminated by '\n'
    location '/path/to/Folder/'
    tblproperties ("skip.header.line.count"="1");

I dont know where I am doing mistake. I am pretty new to hive. So kindly help me.

amitava amitava · Accepted Answer · 2016-05-11T13:26:07

As I know, when you load data from other non-hive data/table to hive, hive takes the order of the fields as it is in source data. So if hive table is partitioned only the last columns in source data can be use as partition.

In your case I am not sure why you are not getting any output, although the output would be wrong as field1 will be field4 and field2 will be field5 in your partitioned table.

The only indirect way (not good) I know is to create the non-partitioned table first as you created, then copy the data from non-partitioned table to partitioned table. If it eats lot of space (although you are going to delete the non-partitioned table later), then you need to change your source data, I guess, to get the partition fields.

External Hive Table with Partition not reading the file

2 Answers