does sqoop export from hdfs to mysql preserve partitions

1
votes

I have created a multi-node hadoop cluster and installed hive on it. Also, on another remote machine I have installed MySQL. I intend to export data stored in HDFS into relational database MySQL. I researched about how this can be done using Sqoop. So I found that I need to create a table in MySQL that has target columns in the same order(as present in Hive), with the appropriate SQL types. And then use the sqoop export command.

My question is: If the table is partitioned in Hive, and if while creating the table in MySQL I partition it accordingly, will the sqoop export command preserve the partitions? My question is similar to sqoop export mysql partition. I want to know if partitioning support has been added to sqoop.

This will help me decide whether to go ahead and install scoop for the task or to use some custom Python scripts that I have written for it.

Thank you.

1
mysqlapachehadoophive
There is a way which may help you, try this: stackoverflow.com/a/17753176/1970125 – Mukesh S

1 Answers

0
votes

Sqoop will work at the JDBC layer when talking to MySQL. It won't be aware of the underlying partitioning, MySQL will handle this as the records are inserted or updated.