0
votes

I am trying to create dynamic partitions in a table.

1) First table is called stock1 where non-partitioned data is residing. Here's the schema :

hive> describe stock1;
id                      int
xcg                     string
stock                   string
sector                  string
country                 string
dt                      string
open                    float
high                    float
low                     float
close                   float
volume                  int
adj_close               float

2) Second table is stock2 where I want the data to be partitioned dynamically.

hive> create table stock2(id int,stock string,sector string,country string,dt string,open float,high float,low float,close float,volume int,adj_close float) PARTITIONED BY (xchng string);

3) Now enabling dynamic partitioning:

set hive.exec.dynamic.partition=true;
set hive.exec.dynamic.partition.mode=nonstrict;

4) loading the data into stock2 from stock1.

insert overwrite table stock2 partition(xchng)select id,stock,sector,country,dt,open,high,low,close,volume,adj_close,t.xcg as xchng from stock1 t;

Now when i check stock2 under the hive folder I only see a default partition

/user/hive/warehouse/stock.db/stock2/xchng=HIVE_DEFAULT_PARTITION

How to rectify so that I get seperate folders for each xchng.

1
You could try enabling dynamic partitioning before you create the second table - OneCricketeer
@cricket_007 : Thanks for quick reply but it didnt resolve the issue. - Deep

1 Answers

0
votes

can you check the all "xcg" values from "stock1". it is possible that at least some of the values or all of the values are not conforming String format. I mean for ex. there can be null or numbers instead of string.