Number of Mapper not able to customized in Sqoop Export

0
votes

Does anyone has experienced on using Custom mapper in Sqoop Export from Hive Table to SQLDatabase before?

I had using the following sqoop command with 33 mappers to performed sqoop export:

sqoop export -Dmapred.job.queue.name=projectname -Dsqoop.export.records.per.statement=1000 --connect "jdbc:sqlserver://svrname;database=dbname" --username 'usrname' --password 'pwd' --hcatalog-database hive_schema_name --hcatalog-table hive_obj_name --table 'SQL_DB_OBJ_NAME' -- --schema SQL_DB_SCHEMA_NAME --fields-terminated-by $'\x01' -m 33 -batch

But as a result, I can only see 4 mappers was using from application master and required a long time to complete due to Huge Amount of Data. So I'm wonder does anyone can help to confirmed whether custom mapper is able to used in sqoop export

1
bigdatasqoop

1 Answers

1
votes

Sqoop export supports number of mappers argument, but it will be ignored in your command. You have to move -- --schema <schema-name> to the end of command since the Sqoop CLI has the following structure:

sqoop TOOL PROPERTY_ARGS SQOOP_ARGS [-- EXTRA_ARGS]