Oozie Hive Action Failed on SerDe

Question

I get an error from oozie hive action workflow :

Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.HiveMain], 
      main() threw exception, org/apache/hadoop/hive/serde2/SerDe
      java.lang.NoClassDefFoundError: org/apache/hadoop/hive/serde2/SerDe

I'm using CDH4.2, HUE 2.2 and JSON-SerDe (rcongiu-hive-json-serde) for my hive table SerDe.

The Hive query script is no problem, I had successfull running it in hive console. I already set the oozie share lib. What make me strongly confuse, I get no error while submit the hive workflow with default table (not using special SerDe) but when I change to my JSON-SerDe table it always failed. the ADD JAR /path-to/serde.jar doesn't work either.

serde.jar must be on HDFS somewhere for Oozie to have any hope of finding it. Maybe you did this already, but I can't be sure from your question. When I needed a jar for an Oozie workflow with a Hive action, copying the jar to HDFS in the Oozie share lib worked for me /user/oozie/share/lib/hive/serde.jar. Doing this or adding <file>other/path/to/jar/on/HDFS/serde.jar</file> to the Hive action should make the jar available to Oozie. Or, it could be that Oozie does see serde.jar, but Hive can't. Unfortunately, I haven't had experience using ADD JAR in Hive script launched by Oozie. — Daniel Koverman
@DanielKoverman yes sir i did put serde.jar on HDFS (sharelib & <file>serde.jar</file>) but i still get error. Strange to me when i'm missing mysql-connector.jar, i just put the JAR file to sharelib (HDFS) and it's work. — mr.bagio
@mr-bagio Oh well, I was hoping I could provide you with an easy fix. Hopefully someone else will come along. — Daniel Koverman
Find the MR job for this launcher, and look at the job.xml - what's the values of the classpath configuration properties (i can't remember off the top of my head what they are called, but look for the jars) — Chris White
Might it relate to this other question about SerDe problems, even though the exception appears to mention a different class as not found? — Randall Whitman

ManojKumar ManojKumar · Accepted Answer · 2018-10-05T07:01:39

I solved this error by placing my Serde JAR file in lib path of application.

Below is my HDFS directory structure:

/user/manoj/app/workflow.xml
/user/manoj/app/lib/json-serde-1.3-jar-with-dependencies.jar

PS: you may have any version of serDe JAR, I used json-serde-1.3-jar-with-dependencies.jar specifically because I found in other answer that this version supports deserialization as well and it worked for me. I was facing "cannot deserialize" error with hive-serde.jar.

Oozie Hive Action Failed on SerDe

1 Answers