HBaseTableCatalog.tableCatalog showing an error- Symbol 'term <none>.yetus.audience' is missing from the classpath. how to resolve this?

0
votes

My input is HBase table. first I am loading the data and function withCatalog will return Dataframe.

def withCatalog(cat: String): DataFrame = {
  sqlContext
    .read
    .options(Map(HBaseTableCatalog.tableCatalog -> cat))
    .format("org.apache.hadoop.hbase.spark")
    .load()
}
val allrulesDF = withCatalog("personDb:employeeTable")
allrulesDF.show()

HBaseTableCatalog line showing below error

  • Symbol term <none>.yetus.audience is missing from the classpath. This symbol is required by <none>. Make sure that term audience is in your classpath and check for conflicting dependencies with -Ylog-classpath. A full rebuild may help if HBaseTableCatalog.class was compiled against an incompatible version of <none>.yetus.

  • Symbol term org.apache.yetus is missing from the classpath. This symbol is required by <none>. Make sure that term yetus is in your classpath and check for conflicting dependencies with -Ylog-classpath. A full rebuild may help if HBaseTableCatalog.class was compiled against an incompatible version of org.apache.

1
scalaapache-sparkapache-spark-sqlhbase

1 Answers

0
votes

It's due to spark cannot load hbase jar.

If you use hbase2.1+, you can find jar likes audience-annotations-*.jar and so on in path $HBASE_HOME/lib/client-facing-thirdparty.

And move these jars to spark jars path.