Datanode and Namenode runs but not reflected in UI

0
votes

I have a small setback in configuring my Master and Slave in Hadoop and I have both my namenode and datanode in Master and Slave up and running.

However the LiveNodes count in the WebUI are not getting reflected but the nodes are running.

I have already tried disabling the firewall and formatted the nodes but I am unable to resolve the same.

Any help would be highly appreciated !!!

Here are the snippets :

Master:

jps command output :

5088 Jps

4446 NameNode

4681 SecondaryNameNode

Slave :

jps command output:

2478 Jps

2410 DataNode

ubuntu@hadoop-master:/usr/local/hadoop/etc/hadoop$ $HADOOP_HOME/bin/hdfs dfsadmin -refreshNodes
16/04/28 02:22:37 WARN ipc.Client: Address change detected. Old: hadoop-master/52.200.230.29:50077 New: hadoop-master/127.0.0.1:50077
refreshNodes: Call From hadoop-master/127.0.0.1 to hadoop-master:50077 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused

Log file of hadoop-slave-1:

2016-04-28 21:23:07,248 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Problem connecting to server: hadoop-master/52.200.230.29:9000
2016-04-28 21:23:12,257 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Problem connecting to server: hadoop-master/52.200.230.29:9000
2016-04-28 21:23:17,265 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Problem connecting to server: hadoop-master/52.200.230.29:9000
2016-04-28 21:23:22,273 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Problem connecting to server: hadoop-master/52.200.230.29:9000
2016-04-28 21:23:27,282 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Problem connecting to server: hadoop-master/52.200.230.29:9000
2016-04-28 21:23:32,291 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Problem connecting to server: hadoop-master/52.200.230.29:9000

Log File of Hadoop-master: 

2016-04-28 21:21:04,002 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from 127.0.0.1
2016-04-28 21:21:04,002 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Rolling edit logs
2016-04-28 21:21:04,002 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Ending log segment 407
2016-04-28 21:21:04,002 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 0 Number of transactions batched in Syncs: 0 Number of syncs: 2 SyncTimes(ms): 22
2016-04-28 21:21:04,003 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 0 Number of transactions batched in Syncs: 0 Number of syncs: 3 SyncTimes(ms): 23
2016-04-28 21:21:04,003 INFO org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Finalizing edits file /usr/local/hadoop/hadoop_data/hdfs/namenode/current/edits_inprogress_0000000000000000407 -> /usr/local/hadoop/hadoop_data/hdfs/namenode/current/edits_0000000000000000407-0000000000000000408
2016-04-28 21:21:04,004 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at 409

netstat -pant command on my master:

ubuntu@hadoop-master:/usr/local/hadoop/etc/hadoop$ netstat -pant
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:50070           0.0.0.0:*               LISTEN      21491/java
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -
tcp        0      0 0.0.0.0:50077           0.0.0.0:*               LISTEN      -
tcp        0      0 127.0.0.1:50078         0.0.0.0:*               LISTEN      21491/java
tcp        0      0 0.0.0.0:9000            0.0.0.0:*               LISTEN      -
tcp        0      0 0.0.0.0:50090           0.0.0.0:*               LISTEN      21726/java
tcp        0      0 172.31.63.189:50070     128.235.8.68:57225      ESTABLISHED 21491/java
tcp        0      0 127.0.0.1:41471         127.0.0.1:50078         TIME_WAIT   -
tcp        0    124 172.31.63.189:22        128.235.8.68:56950      ESTABLISHED -
tcp        0      0 172.31.63.189:50070     128.235.8.68:57224      ESTABLISHED 21491/java
tcp        0      0 172.31.63.189:50070     128.235.8.68:57223      ESTABLISHED 21491/java
tcp        0      0 172.31.63.189:22        128.235.8.68:57084      ESTABLISHED -
tcp        0      0 172.31.63.189:22        58.218.204.215:39402    ESTABLISHED -
tcp        0      0 172.31.63.189:50070     128.235.8.68:57227      ESTABLISHED 21491/java
tcp        0      0 172.31.63.189:50070     128.235.8.68:57228      ESTABLISHED 21491/java
tcp        0      0 172.31.63.189:50070     128.235.8.68:57226      ESTABLISHED 21491/java
tcp6       0      0 :::22                   :::*                    LISTEN      -
tcp6       0      0 :::50077                :::*                    LISTEN      -
tcp6       0      0 :::9000                 :::*                    LISTEN      -
1
hadoophadoop2bigdata
Please paste few lines of namenode and datanode logs - neeraj
Hi, I have edited my question with the log giles - Harinarayanan Mohan
namenode seems like up. But datanodes are failing to make connection to namenode . Have you modified the port no of namenode? I see 9000, while default is 8020. Verify all configs have correct address - neeraj

1 Answers

0
votes

Connection refused I can see this error from your post. I guess you need to do 3 things

  1. make sure 50077 port is listen by a process and it is your hadoop process
  2. make sure it is access able using some tools like telnet
  3. besides firewall. selinux can also affect access. So shut it down and restart your service and try again