How to connect spark application to secure HBase with Kerberos

Question

I`m trying to connect a Spark application to HBase with Kerberos enabled. Spark version is 1.5.0, CDH 5.5.2 and it's executed in yarn cluster mode.

When HbaseContext is initialized, it throws this error:

ERROR ipc.AbstractRpcClient: SASL authentication failed. The most likely cause is missing or invalid credentials. Consider 'kinit'. javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]

I have tried to do the authentication in the code, adding:

UserGroupInformation.setConfiguration(config)
UserGroupInformation.loginUserFromKeytab(principalName, keytabFilename)

I distribute the keytab file with --files option in spark-submit. Now, the error is:

java.io.IOException: Login failure for [email protected] from keytab krb5.usercomp.keytab: javax.security.auth.login.LoginException: Unable to obtain password from user ...

Caused by: javax.security.auth.login.LoginException: Unable to obtain password from user at com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:856)

Is this the way to connect to Kerberized HBase from a Spark app?

have you missed conf.set("hadoop.security.authentication", "Kerberos"); ? — Ram Ghadiyaram

Ram Ghadiyaram Ram Ghadiyaram · Accepted Answer · 2016-05-04T14:40:03

please see the example configuration like below if you are missing anything like hadoop.security.authentication

val conf= HBaseConfiguration.create()
conf.set("hbase.zookeeper.quorum", "list of ip's")
conf.set("hbase.zookeeper"+ ".property.clientPort","2181");
conf.set("hbase.master", "masterIP:60000");
conf.set("hadoop.security.authentication", "kerberos");

How to connect spark application to secure HBase with Kerberos

3 Answers