Using JsonSerDe in a hive script in an on-demand hdinsight cluster

Question

I have written a hive script that works fine in an existing HDInsight cluster. But when I instantiate the script via the Azure Data Factory in an on demand cluster, I get the following error:

Caused by: java.lang.ClassNotFoundException: Class org.apache.hive.hcatalog.data.JsonSerDe not found
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2214)
at org.apache.hadoop.hive.ql.plan.PartitionDesc.getDeserializer(PartitionDesc.java:143)
at org.apache.hadoop.hive.ql.exec.MapOperator.getConvertedOI(MapOperator.java:316)
... 19 more

I use JsonSerDe to load data from a JSON file into a hive table. In my existing cluster the class is present without any additional configuration. Is it necessary to specify the JsonSerDe JAR file explicitly for an on demand cluster and if so how is this possible?

Answer 1

You will receive this error message, when you are missing the Jar file.

You can add the Jar file in the ADF Hive activity as shown.

Go to the activity => File Path => Browse Local and add the Jar file.