unsuccess sqoop-import, hive can't show table

1
votes

I try to import database from mysql to Hive with Hadoop with automatically create table and load data to hive with " --hive import " command from sqoop.

I use command bellow to execute import with sqoop

./sqoop-import --connect jdbc:mysql://localhost/paman -table bibis -m 1 -hive-import

when execute this command :

hadoop@dewi:/opt/sqoop/bin$ ./sqoop-import --connect jdbc:mysql://localhost/paman -table bibis -m 1 -hive-import
12/06/11 16:08:47 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override
12/06/11 16:08:47 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc.
12/06/11 16:08:47 INFO tool.CodeGenTool: Beginning code generation
12/06/11 16:08:47 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver
java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver
    at com.cloudera.sqoop.manager.SqlManager.makeConnection(SqlManager.java:597)
    at com.cloudera.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:51)
    at com.cloudera.sqoop.manager.MySQLManager.execute(MySQLManager.java:201)
    at com.cloudera.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:177)
    at com.cloudera.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:161)
    at com.cloudera.sqoop.orm.ClassWriter.generate(ClassWriter.java:908)
    at com.cloudera.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82)
    at com.cloudera.sqoop.tool.ImportTool.importTable(ImportTool.java:337)
    at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:423)
    at com.cloudera.sqoop.Sqoop.run(Sqoop.java:144)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
    at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:180)
    at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:218)
    at com.cloudera.sqoop.Sqoop.main(Sqoop.java:228)
hadoop@dewi:/opt/sqoop/bin$

what's wrong with my sqoop command ? or are there other configuration in sqoop or hive ?

Help me please

4
hadoophivesqoop

4 Answers

1
votes

First of all, sqoop checks the jars in location

$SQOOP_HOME/lib

You have to add mysql-connector jar to this location. And you have to include --hive-table parameter in your sqoop import command as follows.

./sqoop-import --connect jdbc:mysql://localhost:<port_number>/paman -table bibis -m 1 -hive-import --hive-table hive_table_name

Here hive_table_name is dynamically created.

0
votes

According to the error logs, it seems sqoop command can't load the MySQL JDBC Driver. You should check out if there is a MySQL JDBC driver in the $SQOOP_HOME/lib path.

Hope this helps.

0
votes

Hadoop needs to know the location of the mysql connector jar.

For example, copy it as follows

sudo cp mysql-connector-java-5.1.24-bin.jar /usr/local/Cellar/hadoop/1.1.2/libexec/lib/

It worked for me.

Another Approach:

Try specifying the extra Jar file using -libjar argument. This is a generic Hadoop argument that Sqoop passes down to the framework.

bin/sqoop import -libjars /path/to/mysql-connector-java-5.1.24-bin.jar --connect "..."
0
votes

I also tried setting the $HADOOP_CLASSPATH variable to point to my SQOOP jar, then everything worked.