Secure Hadoop - Datanode cannot connect with namenode

1
votes

I am using hadoop-2.6.0 and created HA enabled cluster with kerberos security in windows platform. Everything works fine if permission is set to false. But when I enable below property,

hdfs-site.xml

<property>
    <name>dfs.permissions</name>
    <value>true</value>
</property>

Datanode cannot connect with the namenode. I am getting the following exception

Exception

2015-05-21 10:44:42,461 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Problem connecting to server: kumar/192.168.3.4:9000
2015-05-21 10:44:46,079 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Problem connecting to server: dinesh/192.168.3.3:9000
2015-05-21 10:44:47,471 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Problem connecting to server: kumar/192.168.3.4:9000
2015-05-21 10:44:51,085 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Problem connecting to server: dinesh/192.168.3.3:9000
2015-05-21 10:44:52,477 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Problem connecting to server: kumar/192.168.3.4:9000

I cannot find the exact root cause for this problem. I need help to solve this.

1
hadoopkerberoshadoop2
Change dfs.permissions to dfs.permissions.enabled. dfs.permissions is deprecated. Refer hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/… - Rajesh N
@RajeshN Both works fine. We can either use dfs.permissions.enabled or dfs.permissions. - Kumar
Could you post your /etc/hosts file? - Rajesh N
I have not set anything in hosts file. Each host added in the same DNS server. So no need to edit in hosts file. - Kumar
@Kumar, try to add your datanode hosts to the hadoop policy - Rengasamy

1 Answers

0
votes

I just changed the default supergroup name to newly created group which has members of all hadoop users. Now all user in that group act as superuser hence it works fine.

<property>
    <name>dfs.permissions.superusergroup</name>
    <value>Hadoopgroup</value>
</property>

Refer superuser