Oozie - Got exception running sqoop: Could not load db driver class: com.mysql.jdbc.Driver

2
votes

I am trying to perform an sqoop export on HDP sandbox 2.1 via Oozie. When I run the Oozie job I get the following java runtime exception.

'>>> Invoking Sqoop command line now >>>

7598 [main] WARN org.apache.sqoop.tool.SqoopTool - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.

7714 [main] INFO org.apache.sqoop.Sqoop - Running Sqoop version: 1.4.4.2.1.1.0-385

7760 [main] WARN org.apache.sqoop.SqoopOptions - Character argument '\t' has multiple characters; only the first will be used.

7791 [main] WARN org.apache.sqoop.ConnFactory - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.

7904 [main] INFO org.apache.sqoop.manager.MySQLManager - Preparing to use a MySQL streaming resultset.

7905 [main] INFO org.apache.sqoop.tool.CodeGenTool - Beginning code generation

7946 [main] ERROR org.apache.sqoop.Sqoop - Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver Intercepting System.exit(1)

I have copied jdbc driver file "mysql-connector-java.jar" to Oozie's shared library folder which I believe is "/usr/lib/oozie/share/lib/sqoop/". I have restarted my sandbox and tried to perform the export with Oozie again and I still get the same error.

The export works perfectly fine when I try performing it only via sqoop, so I presume Oozie needs its own set of drivers.

My question is, which Oozie directory am I suppose to copy my jdbc drivers to?

If you guys think I'm doing something wrong or you need further information, please let me know.

Thank you for your time.

1
mysqlhadoopsqoopooziehortonworks-data-platform
"which I believe is" part of the CLASSPATH. Apparently that assumption is wrong.... - duffymo
possible duplicate of Oozie + Sqoop: JDBC Driver Jar Location - duffymo
Thanks for your response @Duffymo, could you please elaborate your first comment regarding the class path? I did check question you mentioned in your second comment, I am still not able to run my job in spite of having the drivers in that folder. - Tech Guy
Read the error messages. I'd check to see if the SQOOP_CONF_DIR environment variable should be set. - duffymo

1 Answers

2
votes

Normally for Oozie the sharelib directory is /user/oozie/share/lib/ on HDFS where "oozie" would be the name of the user which is used to start the Oozie Server. I don't know what that is in case of HDP sandbox 2.1 , but you can use ps command to figure that out.

And for jars needed for sqoop action, I think you should copy the jar to /user/oozie/share/lib/sqoop/ folder.