sqoop import to hive table error

Question

Hello i try to insert as400 table to hive table, so i create a hive table with the same schema of the as400 table but i add a columns processedate (timestamp) this is my hive table:

CREATE external TABLE IF NOT EXISTS default.AS400TEST (
One INT,
Two STRING,
..
N   INT             
)
PARTITIONED BY (processedate TIMESTAMP)
STORED AS  TEXTFILE 
location  '/tmp/AS400TEST;

I run this sqoop command:

sqoop import --verbose --driver $DRIVER_CLASS --connect $URL --username $LOGIN --password $PASSWORD \
--table $TABLE \
--target-dir $DIR \
-m 1 --hive-import --hive-table default.AS400TEST \
--hive-partition-key "processedate"

My problem is that the map remains on 0% for more than 30 min and i kill the job. did i need to change Something?

Another question: how to insert the column timestamp with sqoop?

Iskuskov Alexander Iskuskov Alexander · Accepted Answer · 2018-04-27T19:03:12

Your table probably is large and one mapper (-m 1) can not complete the task within a reasonable time. Try to increase this parameter (see details in documentation).

To insert timestamp column use --map-column-hive 'timestamp_column_name=TIMESTAMP'.

sqoop import to hive table error

1 Answers