0
votes

I just built a Hortonworks cluster with Hive and Spark Thrift Server on port 10016 (default) and tested SELECT statement via beeline to STS

https://spark.apache.org/docs/1.6.0/sql-programming-guide.html#running-the-thrift-jdbcodbc-server

beeline> !connect jdbc:hive2://localhost:10016/default
select * from table1;

I could get the results and everything is fine. But my question is how come the queries don't show up in Spark History Server UI at all? How do I know if it's actually Spark doing the computation work vs. pass thru HiveServer2?

UPDATE 1

This is what I have and log is already enabled by default:

enter image description here

1

1 Answers

0
votes

If you are using connection with port 10016 - it's Spark. Please check property spark.eventLog.enabled is set to true (it have to be in $SPARK_HOME/conf/spark-defaults.conf).