Impala paritioned table with hdfs

Question

I have data stored in hdfs in the below format and inserted this data in impala partition table using "alter table add partition" command.

/user/impala/subscriber_data/year=2013/month=10/day=01 /user/impala/subscriber_data/year=2013/month=10/day=02

and everything is working fine.

Now I have a new data with month and year as 10 and 01. Now I need to process this data and append this data into existing hdfs directory(year=2013/month=10/day=01).

When I try to process and insert into hdfs directory, its giving error as output directory already exists.

Is there any way to append the new data into existing hdfs directory without deleting the existing directory?

Also, how to insert the new data into existing partition using impala? (I have only table with partition on year,month,day).

Answer 1

to insert into existing partition, you have to drop the existing partition, and add it back with all the files that make up that partition including your new data.