How to use Sqoop libjars option?

1
votes

I am using sqoop import tool for importing data into HBASE table. And I have created a custom PutTransformer class and I have suplied the jar and the property to use custom transformer, but it is not considering the custom transformer class.

Can any one help me on this.

Sqoop Command:

sqoop import -libjars /home/ghk/hbase/poc/customsqoop.jar -D sqoop.hbase.insert.put.transformer.class=com.sqoop.transformer.CustomPutTransformer --connect 'jdbc:oracle:thin:@//XX.XY.XA.XZ:1521/a701' --table XYZ.TEMP_HBASE_TYPE2 --columns 'TID, HBASE_TYPE, DESCRIPTION, IS_DELETED, CREATED' --hbase-create-table --hbase-table 'TEMP_HBASE_TYPE2' --column-family CLMN --hbase-row-key 'TID' --fetch-size '10000' --class-name 'HBaseType' --split-by 'TID' --username xxx--password xxxyy --compress --compression-codec org.apache.hadoop.io.compress.SnappyCodec --num-mappers 3 --outdir /home/ghk/hbase/poc/SqoopClasses

Regards, GHK.

1
sqoop
its working now. Thank you. – GHK
Would you mind sharing logs or any error exceptions that you might be getting? – Jarek Jarcec Cecho

1 Answers

0
votes

Use the environment variable "SQOOP_USER_CLASSPATH" defined in /etc/sqoop/conf/sqoop-env.sh, and do not use the libjars argument.

$ grep USER /etc/sqoop/conf/sqoop-env.sh 
export SQOOP_USER_CLASSPATH="`ls ${HIVE_HOME}/lib/libthrift-*.jar 2> /dev/null`:${SQOOP_USER_CLASSPATH}"
$ export SQOOP_USER_CLASSPATH=/custom/lib01.jar:/custom/lib02.jar
$ sqoop import ...