Access Hadoop with kerberos failed

Question

I install hadoop and kerberos, but when i exec hadoop fs -ls / error has bean occured.

[dannil@ozcluster06 logs]$ hadoop fs -ls /
16/09/13 11:34:39 WARN ipc.Client: Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate    failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
ls: Failed on local exception: java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]; Host Details : local host is: "localhost/127.0.0.1"; destination host is: "192.168.168.46":9000;

i can see the datanode and namenode has start up by jps

  20963 DataNode
  21413 SecondaryNameNode
  20474 NameNode
  22906 Jps

i add principal hdfs/[email protected] and HTTP/[email protected],then i use xst -norandkey -k hdfs.keytab hdfs/[email protected] HTTP/[email protected] to generate a hdfs.keytab

kadmin.local:  listprincs
HTTP/[email protected]
K/[email protected]
dannil/[email protected]
hdfs/[email protected]
kadmin/[email protected]
kadmin/[email protected]
kadmin/[email protected]
kiprop/[email protected]
krbtgt/[email protected]

then i exec kinit -kt /home/dannil/hadoop-2.7.1/hdfs.keytab hdfs/oz.flex

i can see my ticket status :

[dannil@ozcluster06 ~]$ klist
Ticket cache: KEYRING:persistent:1000:krb_ccache_4h73plA
Default principal: hdfs/[email protected]

Valid starting       Expires              Service principal
2016-09-13T10:47:06  2016-09-14T10:47:06  krbtgt/[email protected]

this is my hadoop config value:

core-site.xml:

fs.defaultFS=hdfs://192.168.168.46:9000
hadoop.security.authentication=kerberos
hadoop.security.authorization=true

hdfs-site.xml:

dfs.replication=1
dfs.permissions=false
dfs.block.access.token.enable=true
dfs.namenode.keytab.file=/home/dannil/hadoop-2.7.1/hdfs.keytab
dfs.namenode.kerberos.principal=hdfs/[email protected]
dfs.namenode.kerberos.internal.spnego.principal=HTTP/[email protected]
dfs.secondary.namenode.keytab.file=/home/dannil/hadoop-2.7.1/hdfs.keytab
dfs.secondary.namenode.kerberos.principal=hdfs/[email protected]
dfs.secondary.namenode.kerberos.internal.spnego.principal=HTTP/[email protected]
dfs.datanode.data.dir.perm=700
dfs.datanode.address=0.0.0.0:61004
dfs.datanode.http.address=0.0.0.0:61006
dfs.datanode.keytab.file=/home/dannil/hadoop-2.7.1/hdfs.keytab
dfs.datanode.kerberos.principal=hdfs/[email protected]
dfs.https.port=50470
dfs.https.address=0.0.0.0:50470
dfs.webhdfs.enabled=true
dfs.web.authentication.kerberos.principal=HTTP/[email protected]
dfs.web.authentication.kerberos.keytab=/home/dannil/hadoop-2.7.1/hdfs.keytab
dfs.http.policy=HTTPS_ONLY
dfs.data.transfer.protection=integrity

How could the error occured ? and what i should do to solve the problem?

Forget about KEYRING: and try the default FILE: implementation for the ticket cache. That one works with the Hadoop "hack" of the Java implementation of the Kerberos client. — Samson Scharfrichter
To enable Kerberos trace logs: export HADOOP_JAAS_DEBUG=true in environment and -Dsun.security.krb5.debug=true in the custom Java options, cf. steveloughran.gitbooks.io/kerberos_and_hadoop/content/sections/… (the whole GitBook is a must-read if you really want to mess with Kerberos and Hadoop) — Samson Scharfrichter

linuxsan linuxsan · Accepted Answer · 2017-12-28T12:06:38

Try to execute the command as 'hdfs' user and check
If that's not working then can you please confirm, whether Kerberos has been enabled in ssh config file, '/etc/ssh/sshd_config', check for the line 'KerberosAuthentication Yes' and also 'GSSAPIAuthentication Yes'. By default it would be commented out, so uncomment that and change it to yes.

And please let us know here, whether it worked out or not. So that it would be helpful for others.

Access Hadoop with kerberos failed

1 Answers