Hadoop - Issues setting up single node cluster

Question

Im trying to setup a Hadoop single node cluster following this article -

http://www.bogotobogo.com/Hadoop/BigData_hadoop_Install_on_ubuntu_single_node_cluster.php

Everthing looks good but when start the hadoop and list all the ports I see few are missing.

These are the ports in my machine. tcp 0 0 0.0.0.0:8088 0.0.0.0:* LISTEN 1001 50434 5765/java
tcp 0 0 0.0.0.0:50010 0.0.0.0:* LISTEN 1001 45587 5461/java
tcp 0 0 0.0.0.0:50075 0.0.0.0:* LISTEN 1001 45594 5461/java
tcp 0 0 0.0.0.0:8030 0.0.0.0:* LISTEN 1001 47365 5765/java
tcp 0 0 0.0.0.0:8031 0.0.0.0:* LISTEN 1001 47353 5765/java
tcp 0 0 0.0.0.0:8032 0.0.0.0:* LISTEN 1001 47373 5765/java
tcp 0 0 0.0.0.0:8033 0.0.0.0:* LISTEN 1001 50861 5765/java
tcp 0 0 0.0.0.0:40641 0.0.0.0:* LISTEN 1001 50843 5885/java
tcp 0 0 0.0.0.0:50020 0.0.0.0:* LISTEN 1001 46697 5461/java
tcp 0 0 0.0.0.0:8040 0.0.0.0:* LISTEN 1001 50850 5885/java
tcp 0 0 0.0.0.0:8042 0.0.0.0:* LISTEN 1001 50854 5885/java
tcp 0 0 0.0.0.0:50090 0.0.0.0:* LISTEN 1001 46221 5610/java

and this the list from the link.

hduser@laptop:~$ netstat -plten | grep java (Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.) tcp 0 0 0.0.0.0:50020 0.0.0.0:* LISTEN 1001 1843372 10605/java
tcp 0 0 127.0.0.1:54310 0.0.0.0:* LISTEN 1001 1841277 10447/java
tcp 0 0 0.0.0.0:50090 0.0.0.0:* LISTEN 1001 1841130 10895/java
tcp 0 0 0.0.0.0:50070 0.0.0.0:* LISTEN 1001 1840196 10447/java
tcp 0 0 0.0.0.0:50010 0.0.0.0:* LISTEN 1001 1841320 10605/java
tcp 0 0 0.0.0.0:50075 0.0.0.0:* LISTEN 1001 1841646 10605/java
tcp6 0 0 :::8040 :::* LISTEN 1001 1845543 11383/java
tcp6 0 0 :::8042 :::* LISTEN 1001 1845551 11383/java
tcp6 0 0 :::8088 :::* LISTEN 1001 1842110 11252/java
tcp6 0 0 :::49630 :::* LISTEN 1001 1845534 11383/java
tcp6 0 0 :::8030 :::* LISTEN 1001 1842036 11252/java
tcp6 0 0 :::8031 :::* LISTEN 1001 1842005 11252/java
tcp6 0 0 :::8032 :::* LISTEN 1001 1842100 11252/java
tcp6 0 0 :::8033 :::* LISTEN 1001 1842162 11252/java

Few important things that I miss is 50070 which is used for the web portal.

Clyde D'Cruz Clyde D'Cruz · Accepted Answer · 2015-12-29T06:44:44

You can check which Hadoop processes are running using jps command.

This will show you the process names like NameNode, ResourceManager etc. preceded by their process Id. eg:

7413 Jps
38754 ResourceManager
37054 NameNode
37338 SecondaryNameNode

This tells you that the Namenode and the Resource manager are alive. You can then use netstat to check which ports these processes are using. eg:

netstat -lnptu |grep 37054
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
tcp        0      0 127.0.0.1:9000         0.0.0.0:*                   LISTEN      37054/java
tcp        0      0 127.0.0.1:50070        0.0.0.0:*                   LISTEN      37054/java

The ports can be changed by setting the appropriate configuration parameter in the correct configuration file.

Hadoop - Issues setting up single node cluster

1 Answers