Hbase Kerberos connection from Spark scala

1
votes

I’m trying to connect to a kerberized secures Hbase cluster from a spark scala shell , below is my code and appreciate any help with the errors . I’m passing hdfs-site.xml, hbase-site.xml , core-site.xml and my keytab in the spark shell using — files 

import org.apache.hadoop.conf.Configuration 
import org.apache.hadoop.hbase.HBaseConfiguration
import org.apache.hadoop.hbase.client.Connection
import org.apache.hadoop.hbase.client.ConnectionFactory 
import org.apache.hadoop.hbase.TableName
import org.apache.hadoop.security.UserGroupInformation

val conf: Configuration = HBaseConfiguration.create()
conf.set("hbase.zookeeper.quorum", "[email protected],[email protected],[email protected]")
conf.set("zookeeper.znode.parent", "/hbase-secure") 
conf.setInt("hbase.client.scanner.caching", 10000)
conf.set("hbase.rpc.controllerfactory.class","org.apache.hadoop.hbase.ipc.RpcControllerFactory")
conf.set("hbase.rpc.controllerfactory.class","org.apache.hadoop.hbase.ipc.RpcControllerFactory")
conf.set("hadoop.security.authentication", "kerberos")
conf.set("hbase.security.authentication", "kerberos")
val userGroupInformation = UserGroupInformation.loginUserFromKeytabAndReturnUGI("[email protected]", "/u/xxxxx/XXXX.keytab")
UserGroupInformation.setLoginUser(userGroupInformation)
val connection: Connection = ConnectionFactory.createConnection(conf)
print(connection)
val admin = connection.getAdmin
val listtables = admin.listNamespaceDescriptors()
]

I see a lot of warning in the process as below 
warning: Class

org.apache.hadoop.hbase.classification.InterfaceAudience not found - continuing with a stub.

Error - ——

WARN AbstractRpcClient: Couldn't setup connection for [email protected] to null

 RpcRetryingCaller{globalStartTime=1541788150382, pause=100, retries=35}, org.apache.hadoop.hbase.MasterNotRunningException: com.google.protobuf.ServiceException: java.io.IOException: Couldn't setup connection for [email protected] to null

  at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:158)
  at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4427)
  at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4416)
  at org.apache.hadoop.hbase.client.HBaseAdmin.listNamespaceDescriptors(HBaseAdmin.java:3123)
  ... 49 elided
Caused by: org.apache.hadoop.hbase.MasterNotRunningException: com.google.protobuf.ServiceException: java.io.IOException: Couldn't setup connection for [email protected] to null
  at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1560)
  at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1580)
  at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1731)
  at org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38)
  at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134)
  ... 52 more
Caused by: com.google.protobuf.ServiceException: java.io.IOException: Couldn't setup connection for [email protected] to null
  at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:228)
  at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:292)
  at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:62896)
  at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(ConnectionManager.java:1591)
  at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1529)
  at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1551)
  ... 56 more
Caused by: java.io.IOException: Couldn't setup connection for [email protected] to null
  at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$1.run(RpcClientImpl.java:665)
  at java.security.AccessController.doPrivileged(Native Method)
  at javax.security.auth.Subject.doAs(Subject.java:422)
  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
  at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.handleSaslConnectionFailure(RpcClientImpl.java:637)
  at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:745)
  at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:889)
  at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:856)
  at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1201)
  at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:218)
  ... 61 more
Caused by: java.io.IOException: Failed to specify server's Kerberos principal name
  at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.<init>(HBaseSaslRpcClient.java:117)
  at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:609)
  at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$600(RpcClientImpl.java:156)
  at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:737)
  at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:734)
  at java.security.AccessController.doPrivileged(Native Method)
  at javax.security.auth.Subject.doAs(Subject.java:422)
  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
  at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:734)
  ... 65 more
1
scalaapache-sparkhbasekerberos
Failed to specify server's Kerberos principal name > missing properties. – Samson Scharfrichter
If you are indeed passing hbase-site.xml to driver and executors, how comes you hard-code so many HBase properties?? – Samson Scharfrichter
I have passed the xml but still having issues which is why I’m manually coding again . – RData

1 Answers

0
votes

Your zookeeper configurations is missing zookeeper port. add,

conf.set("hbase.zookeeper.property.clientPort", "2181")

I would like you to suggest you few other things,

  1. After this you might have problems with spark configurations. Copy hdfs-site.xml, hbase-site.xml, core-site.xml and yarn-site.xml (If you are using yarn) to spark conf folder.

  2. Add resources to hbase configuration object

    conf.addResource("/path/to/hbase-site.xml");

  3. Set java properties

    //Point to the krb5.conf file. Enable Kerberos debug.
System.setProperty("java.security.krb5.conf", "/etc/krb5.conf");
System.setProperty("sun.security.krb5.debug", "true");

Hope this would help...