Issue connecting to Hive using JDBC with Kerberos authentication

Question

I am trying to write a program to connect to Hive using Jdbc Driver with Kerberos authentication. My program is as folloows :

public static void main(String a[])
{

    ...

    connectionString_ = "jdbc:hive2://cdh-542-kerberos.domain.com:10000/default;principal=hive/cdh-542-kerberos.domain.com@REALMDOMAIN";

    UserGroupInformation ugi = createUgi();

    connection = ugi.doAs(new PrivilegedExceptionAction<Connection>() {
        public Connection run() throws Exception {
            Connection connection = null;
            Class.forName(jdbcDriverClass_);
            connection =  DriverManager.getConnection(connectionString_);
            return connection;
        }
    });
    ... 
}



public static UserGroupInformation createUgi() 
{
    try{
        UserGroupInformation ugi = null;
        
        String principal = "hive/cdh-542-kerberos.domain.com@REALMDOMAIN"
        String keyTabLocation = "hive.keytab"
        
        logger.debug("principal:" + principal);
        logger.debug("keyTabLocation:" + keyTabLocation);
        ugi =  createkerbUser( principal, keyTabLocation);
        return ugi;
    }
    catch(Exception e)
    {
        throw new BDEToolSecurityException(BDEToolSecurityException.SECURITYEXCPETION, e);
    }
}

public static UserGroupInformation createkerbUser( String principal, String keyTabFilePath) 
{
    UserGroupInformation app_ugi = UserGroupInformation.loginUserFromKeytabAndReturnUGI(principal, keyTabFilePath);
    String user = getUserfromPrincipal(principal);
    
    if(user.trim().length() > 0){
        UserGroupInformation proxy_ugi = UserGroupInformation.createProxyUser(user, app_ugi);
        return proxy_ugi;
    }
    return app_ugi;
}

private static String getUserfromPrincipal(String principal)
{
    String user = Constants.emptyString;
    
    if(principal.contains(Constants.fSlash)){
        String[]  tokens = principal.split(Constants.fSlash);           
        if(tokens.length > 1){
            user = tokens[0];
        }            
    }       
    return user;
}

This code is failing when it is trying to ugi.doAs() ... with below exception

java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://cdh-542-kerberos.informatica.com:10000/default;principal=hive/cdh-542-kerberos.informatica.com@INFAQAKERB: GSS initiate failed
    at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:215)
    at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:163)
    at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
    at java.sql.DriverManager.getConnection(DriverManager.java:571)
    at java.sql.DriverManager.getConnection(DriverManager.java:233)
    at com.informatica.gcs.tools.bde.connectivity.tool.hive.HiveJdbcTest$1.run(HiveJdbcTest.java:148)
    at com.informatica.gcs.tools.bde.connectivity.tool.hive.HiveJdbcTest$1.run(HiveJdbcTest.java:144)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
    ... 6 more
Caused by: org.apache.thrift.transport.TTransportException: GSS initiate failed
    at org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232)
    at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316)
    at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
    at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
    at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
    at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
    at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:190)
    ... 15 more

Both the keytab and princiapl seems to be correct. I can successfully run the command

kinit -k -t hive.keytab hive/cdh-542-kerberos.informatica.com@INFAQAKERB

Can anybody help me further on this. I am stuck for almost a week on this problem. I can provide additional information, if required.

GSS initiate failed this error generally occurs when your kerboros ticket gets expired. Try re validating the token each time you run the java code. — K S Nidhin
I have already tried regenerating ticket using kninit command. But no luck. — Fayaz
What flavor of JVM are you using? And what kind of encryption is expected by the KDC? (spoiler: the Oracle/Sun JRE does not support AES256 by default, you must download the "unlimited strength cryptography") — Samson Scharfrichter
Did you try the JAAS trace flag -Djava.security.debug=gssloginconfig,configfile,configparser,logincontext to debug your issue? — Samson Scharfrichter

Tagar Tagar · Accepted Answer · 2016-05-16T22:12:26

It's not a missing/expired ticket problem. Otherwise error stack will read also "Caused by GSSException: No valid credentials provided" down right after "GSS initiate failed".

Possible problems:

It seems that the server where you're trying to connect to (Hive Server 2, port 10000) and your client machine (that makes jdbc: hive connection) are in different domains. Is it the case?
To confirm that your server and client can authenticate in that one domain, and its domain configurations are correct (krb5.conf etc.), do
```
$ kinit -k -t hive.keytab hive/cdh-542-kerberos.informatica.com@INFAQAKERB (like you did)
$ klist 
```
- do this on server and client side and copy-paste result here.
"org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:215) " can give a clue that it might be a firewall/domain configuration issue from HS2 server to domain controller server. Although you can create ticket on client side, HS2 server can't validate it.
On your code snippets - check out Examples of connecting to Kerberos Hive in JDBC.

It seems your code is at least missing:
```
UserGroupInformation.setConfiguration(conf);
```
not sure though if it's relevant for your use case...

Keep us updated if you are able to resolve this.

Issue connecting to Hive using JDBC with Kerberos authentication

2 Answers