Zeppelin : Not Showing Hive Database/tables in HDP3.0

0
votes

I have installed Hortonworks hdp3.0 and configured Zeppelin as well.

When I am running spark or sql, Zeppelin is only showing me default database(This is the default database from Spark which has location as '/apps/spark/warehouse', not the default database of Hive). This is probably because hive.metastore.warehouse.dir property is not set from hive-site.xml and zeppelin is picking this from Spark config (spark.sql.warehouse.dir).

I had similar issue with spark as well and it was due to hive-site.xml file on spark-conf dir, I was able to resolve this by copying hive-site.xml from hive-conf dir to spark-conf dir.

I did the same for Zeppelin as well, copied hive-site.xml in zeppelin dir(where it has zeppelin-site.xml and also copied in zeppelin-external-dependency-conf dir.

But this did not resolve the issue.

1
apache-sparkhadoophivehortonworks-data-platformapache-zeppelin
did you enable Hive support when creating a Spark session? – Vamsi Prabhala
Zeppelin doesn't use a Hive site, it only takes the one available to Spark... Have you tried a JDBC Connector to Hive? – OneCricketeer
@vamsiprabhala I have create spark session by enabling hive support through enableHiveSupport(), and even tried setting spark.sql.warehouse.dir config property. but this did not help. import org.apache.spark.sql.SparkSession val spark = SparkSession.builder.appName("Spark Hive Example").config("spark.sql.warehouse.dir", "/apps/hive/db").enableHiveSupport().getOrCreate() – Shan
@cricket_007 - That's useful info, but I can view all hive databases through spark. it's only in Zeppelin (through spark2) I am not able to access Hive databases. Additionaly Zeppelin is not letting me choose programming language, it by default creates session with scala. Any help on this will be highly appreciated. – Shan
Using the Hive context is a default value, and you can change the interpreter in any cell with a magic command. zeppelin.apache.org/docs/latest/interpreter/spark.html I'm not sure where you're looking for the databases at, but I've seen when using Spark thriftserver that only the default will display, but that won't stop you from running queries on others. Did you reboot Zeppelin after moving your XML files around? – OneCricketeer

1 Answers

1
votes

After copying hive-site.xml from hive-conf dir to spark-conf dir, I restarted the spark services that reverted those changes, I copied hive-site.xml again and it's working now.

cp /etc/hive/conf/hive-site.xml /etc/spark2/conf