Hadoop UI shows only one Datanode

Question

I've started hadoop cluster composed of on master and 4 slave nodes.

Configuration seems ok:

hduser@ubuntu-amd64:/usr/local/hadoop$ ./bin/hdfs dfsadmin -report

When I enter NameNode UI (http://10.20.0.140:50070/) Overview card seems ok - for example total Capacity of all Nodes sumes up.

The problem is that in the card Datanodes I see only one datanode.

Answer 1

I came across the same problem, fortunately, I solved it. I guess it causes by the 'localhost'.

1. Config different name for these IP in /etc/host
2. Remember to restart all the machines, things will go well.

Answer 2

It's because of the same hostname in both datanodes. In your case both datanodes are registering to the namenode with same hostname ie 'localhost' Try with different hostnames it will fix your problem.

in UI it will show only one entry for a hostname.

in "hdfs dfsadmin -report" output you can see both.

Answer 3

The following tips may help you

1. Check the core-site.xml and ensure that the namenode hostname is correct
2. Check the firewall rules in namenode and datanodes and ensure that the required ports are open
3. Check the logs of datanodes
4. Ensure that all the datanodes are up and running

Answer 4

As @Rahul said the problem is because of the same hostname

change your hostname in /etc/hostname file and give different hostname for each host and resolve hostname with ip address /etc/hosts file

then restart your cluster you will see all datanodes in Datanode information tab on browser

Answer 5

I have the same trouble because I use ip instead of hostname, [hdfs dfsadmin -report] is correct though it is only one[localhost] in UI. Finally, I solved it like this:

<property>
       <name>dfs.datanode.hostname</name>                   
       <value>the name you want to show</value>
</property>

you almost can't find it in any doucument...

Answer 6

Sorry, feels like it's been a time. But still I'd like to share my answer: the root cause is from hadoop/etc/hadoop/hdfs-site.xml: the xml file has a property named dfs.datanode.data.dir. If you set all the datanodes with the same name, then hadoop is assuming the cluster has only one datanode. So the proper way of doing it is naming every datanode with a unique name:

Regards, YUN HANXUAN

Answer 7

Your admin report looks absolutely fine. Please run the below to check the HDFS disk space details.

"hdfs dfs -df /"

If you still see the size being good, its just a UI glitch.