hadoop slave cannot connect to master:8031

0
votes

I'm a total hadoop newbee. I set up hadoop on two machines master and slave following this tutorial (I obtained the same error following this other tutorial).

Problem: After starting dfs and yarn, the only node appearing on localhost:50070 is the master, even if the right processes are running on the master (NameNode, DataNode, SecondaryNameNode, ResourceManager) and on the slave (DataNode).

The nodemanager log of the slave reports: INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.14:8031. Already tried 10 times.

Note that:

  1. I have already edited yarn-site.xml following this thread.
  2. I have disabled the firewall on the master
  3. I ran netstat -anp | grep 8031 on the master and confirmed that there are a couple of processes listening on port 8031 using tcp.
1
hadoop
It appears that the ResourceManager is not running on your Master, can you view the logs of the ResourceManager on the master? – AbdulRahman AlHamali
It is running -- sorry for not specifying it. I've edited the question. – user3537808
Ok but what do the logs of the resource manager show? – AbdulRahman AlHamali

1 Answers

0
votes

I suffered from the same problem, and these steps may help.

  1. Make sure your namenodes can negotiate with your datanodes.
hadoop@namenode-01:~$ jps
8678 NameNode
9530 WebAppProxyServer
9115 ResourceManager
8940 SecondaryNameNode
9581 Jps

hadoop@datanode-01:~$ jps
8592 NodeManager
8715 Jps
8415 DataNode
  1. Check your nodemanager-datanode-01.log from $HADOOP_HOME/logs reports.
INFO org.apache.hadoop.ipc.Client: Retrying connect to server: namenode-01/192.168.4.2:8031. Already tried 8 time(s);
  1. Check if your yarn.resourcemanager.resource-tracker.address listen on ipv6 instead of ipv4.
hadoop@namenode-01:~$ netstat -natp
tcp6       0      0 127.0.2.1:8088          :::*                    LISTEN      9115/java
tcp6       0      0 127.0.2.1:8030          :::*                    LISTEN      9115/java
tcp6       0      0 127.0.2.1:8031          :::*                    LISTEN      9115/java
tcp6       0      0 127.0.2.1:8032          :::*                    LISTEN      9115/java
tcp6       0      0 127.0.2.1:8033          :::*                    LISTEN      9115/java
  1. If your yarn addresses listen on ipv6, maybe you should disable ipv6 first.
sysctl -w net.ipv6.conf.all.disable_ipv6=1
sysctl -w net.ipv6.conf.default.disable_ipv6=1
  1. Finally, the default yarn addresses are like this:
yarn.resourcemanager.resource-tracker.address   ${yarn.resourcemanager.hostname}:8031
  1. Check your /ect/hosts to avoid misconfigurations.
hadoop@namenode-01:~$ cat /etc/hosts
127.0.2.1 namenode-01 namenode-01 (for example, this line config hostname as 127.0.2.1, delete this line)
192.168.4.2 namenode-01
192.168.4.3 datanode-01
192.168.4.4 datanode-02
192.168.4.5 datanode-03