Where are the Hive logs generated by a MongoDB Hadoop connector failure in HDP Sandbox?

0
votes

I'm facing a strange error I can't find in any logs on my Azure Hortonworks Sandbox. I'm trying to execute the following how-to to try the Mongodb integration with Hadoop (Hive) : https://www.mongodb.com/blog/post/using-mongodb-hadoop-spark-part-2-hive-example, but when I try to create a table in Mongo with Hive, I get the following error (using hive CLI, beeline and Ambari Hive view on both MR2 and Tez processor) : 

Error: Error while processing statement:  FAILED: Execution Error, 
return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.com/mongodb/BasicDBObject (state=08S01,code=1)

Well, googling this message I found people quoting some java logs describing what the real exception is, and conclude (most of the time) the mongodb-java-driver jar were missing to the classpath.

My first question : Where are these logs describing the real exception? Because looking in /var/log/hive*/*.log and Yarn ResourceManagerUI -> logs, I can't see anything but "InterruptedException", "ERROR applicationhistoryservice.ApplicationHistoryServer (LogAdapter.java:error(69)) - RECEIVED SIGNAL 15: SIGTERM" and so on... But nothing really useful.

Next, I added the mongo-hadoop-core and mongo-hadoop-hive jars (built from github against my sandbox version specifications) AND the version 3.3.0 of mongodb-java-driver in a local path, specified in the section hive.aux.jars.path of the hive-site.xml, and finally restarted my hive service (so, including hive-server2)... But no way to get this working, and no way to get more info to understand why it's not running...

Am I forgetting something? Thanks for your help!

1
mongodbhadoophivehortonworks-data-platformhortonworks-sandbox

1 Answers

0
votes

Well, I finally found what was my problem.

The jars bson-$MongoVersion.jar and mongodb-driver-core-$MongoVersion.jar were missing.

Even if the connector documentation specifies we need to include mongo-hadoop-core-$connectorVersion.jar, mongo-hadoop-hive-$connectorVersion.jar and mongodb-driver-$mongoJavaDriver.jar to the hadoop classpath, it seems that it needs bson and mongodb-driver-core jars, too.

Finally, logs are located here : /var/log/hive/hiveserver2.log. I don't understand how I could miss such a message reading theses logs, but well, here it was.