Can we export special characters using sqoop?

1
votes

I'm trying to export one of the tables from hive to MySQL using sqoop export. The hive table data contains the special characters.

My hive "special_char" table data:

1   じゃあまた
2   どうぞ

My Sqoop Command:

 sqoop export --verbose --connect jdbc:mysql://xx.xx.xx.xxx/Sampledb --username abc --password xyz --table special_char --direct --driver com.mysql.jdbc.Driver  --export-dir /apps/hive/warehouse/sampledb.db/special_char --fields-terminated-by ' '

After using the above sqoop export command, the data is stored in the form of question marks (???) instead of actual message with special characters.

MySql "special_char" table:

id  message
1    ?????
2    ???

Can anyone please help me out,in storing the special characters instead of question marks (???).

1
mysqlhivesqoop
Did you try row level inserting same characters directly in SQL without Sqoop? If yes, is that working as expected? – Rohit Nimmala
Nope.. I'll try row level inserting same characters directly in SQL without Sqoop. – Intrigue777
Yes, I'm able to insert the special characters into MYSQL directly – Intrigue777

1 Answers

0
votes

Specify proper encoding and charset in the JDBC URL as below:

jdbc:mysql://xx.xx.xx.xxx/Sampledb?useUnicode=true&characterEncoding=UTF-8

sqoop export --verbose --connect jdbc:mysql://xx.xx.xx.xxx/Sampledb?useUnicode=true&characterEncoding=UTF-8 --username abc --password xyz --table special_char --direct --driver com.mysql.jdbc.Driver  --export-dir /apps/hive/warehouse/sampledb.db/special_char --fields-terminated-by ' '

Please verify charset encoding for Japanese characters and use proper one.

Reference: https://community.hortonworks.com/content/supportkb/198290/native-sqoop-export-from-hdfs-fails-for-unicode-ch.html