Import specific partitions from a table into hadoop

0
votes

I am using sqoop to import data into hadoop. I am using below command to import a table into hadoop.

 sqoop import \
 --connect jdbc:oracle:thin:@172.16.XXX.XXX:15XX:TABS \
 --username user \
 --password pass \
 --table TABS.ADM_LOG0009 \
 --target-dir /hadoop/hptabs/recharge

Now I have another table named IVM_IMAGE, which is a partitioned table. I want to import only 3 partitions from Oracle to hadoop. Please help me to write command for this.

  • Oracle table name: TABS.IVM_IMAGE
  • Partitions: LESS20161101, LESS20161001, LESS20160901

  • Partiton key: BILLDATE

  • HIVE table name: hptabs.IVM_IMAGE

Table properties:

PARTITIONED BY (BILLDATE TIMESTAMP) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LOCATION 'hdfs:/hadoop/hptabs/usage';

1
oraclehadoophdfssqoop
You mean Hive table name not Hadoop table name. Right? - Dev
Yes Hive table name. - Shariar Imtiaz
this will help. - Dev
Thanks. it works at last. - Shariar Imtiaz
Good to know..! You can self-answer (if you want) to help the community. - Dev

1 Answers

2
votes

To import specific partitions i need to add "-Doraoop.import.partitions='"LESS20161101","LESS20161001","LESS20160901",'" to my sqoop command

sqoop import \
-Doraoop.disabled=false \
-Doraoop.import.partitions='"LESS20161101","LESS20161001","LESS20160901",' \
--connect jdbc:oracle:thin:@172.16.XXX.XXX:15XX:TABS \
--username user \
--password password \
--table TABS.IVM_IMAGE \
--target-dir /hadoop/hptabs/usage/ \
-m 1