Setting hive.metastore.warehouse.dir ('null') to the value of spark.sql.warehouse.dir ('/user/hive/warehouse')

Question

I am using spark 2.2 and oozie . And I have schedule the spark-action in the oozie through shell-script i.e I am calling shell script in oozie which contains the spark-submit command.

But when I am running the job , it gives me below INFO statement: Setting hive.metastore.warehouse.dir ('null') to the value of spark.sql.warehouse.dir ('/user/hive/warehouse').

Now as the property - hive.metastore.warehouse.dir = null , my job is not able to identify the tables and databases .

How to fix it or set this property . I tried setting the below property in hive-site.xml , but still facing same error

    <property>
        <name>hive.metastore.warehouse.dir</name>
        <value>/user/hive/warehouse</value>
    </property>

Aakash Agrawal Aakash Agrawal · Accepted Answer · 2019-09-04T07:20:05

As the above was run using oozie , the error was that the hive.xml has not been passed along with the job .

Passing the correct hive.xml resolved the above issue.

Setting hive.metastore.warehouse.dir ('null') to the value of spark.sql.warehouse.dir ('/user/hive/warehouse')

1 Answers