Hadoop: resourcemanager doesn't run on localhost

Question

So I can't manage to access http://localhost:8088/ on hadoop 3.1.1

Here is what I did:

bin/hdfs namenode -format
sbin/start-dfs.sh
bin/hdfs dfs -mkdir /user
bin/hdfs dfs -mkdir /user/username

The web interface for the NameNode works but it doesn't for the Resource Manager.

core-site.xml:

<configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://localhost:9000</value>
    </property>
</configuration>

hdfs-site.xml:

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
</configuration>

mapred-site.xml:

    <configuration>
        <property>
            <name>mapreduce.framework.name</name>
            <value>yarn</value>
        </property>
    </configuration>
    
    <configuration>
        <property>
            <name>mapreduce.application.classpath</name>
            <value>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*</value>
        </property>
    </configuration>

yarn-site.xml:

<configuration>
    <property>
        <name>yarn.resourcemanager.address</name>
        <value>127.0.0.1:8032</value>
    </property>
    <property>
        <name>yarn.resourcemanager.scheduler.address</name>
        <value>127.0.0.1:8030</value>
    </property>
    <property>
        <name>yarn.resourcemanager.resource-tracker.address</name>
        <value>127.0.0.1:8031</value>
    </property>
    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
    <property>
        <name>yarn.nodemanager.env-whitelist</name>
        <value>JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME</value>
    </property>
</configuration>

.bash_profile

export PATH="/usr/local/sbin:$PATH"
    
export SCALA_HOME=/usr/local/scala
export JAVA_HOME=/Library/Java/JavaVirtualMachines/openjdk-11.0.1.jdk/Contents/Home
export SPARK_HOME=/usr/local/spark
    
export HADOOP_HOME=/usr/local/Cellar/hadoop/3.1.1/libexec/
export HADOOP_CONF_DIR=/usr/local/Cellar/hadoop/3.1.1/libexec/etc/hadoop
    
export PATH=$PATH:/usr/local/hadoop/bin
export PATH=$PATH:/usr/local/spark/bin
export PATH=$PATH:/usr/local/scala/bin
    
export HADOOP_MAPRED_HOME=$HADOOP_HOME
    
export HADOOP_COMMON_HOME=$HADOOP_HOME
    
export HADOOP_HDFS_HOME=$HADOOP_HOME
    
export YARN_HOME=$HADOOP_HOME
    
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
    
export HADOOP_OPTS=-Djava.library.path=$HADOOP_HOME/lib
    
export PATH=$HADOOP_HOME/bin:$PATH
    
export PATH=$HADOOP_HOME/sbin:$PATH

The problem is when I run: sbin/start-yarn.sh here is the result
Starting resourcemanagers on []
Starting nodemanagers

shouldn't it say: Starting resourcemanagers on [localhost] ?

Ajay Kharade Ajay Kharade · Accepted Answer · 2019-01-14T21:16:59

By default value for hostname from documentation is 0.0.0.0 and not localhost, if you want to configure it explicitly below are the properties and here its default values provided, you can override same.

yarn.resourcemanager.hostname 0.0.0.0 The hostname of the RM. yarn.resourcemanager.address ${yarn.resourcemanager.hostname}:8032 The address of the applications manager interface in the RM.

Hadoop: resourcemanager doesn't run on localhost

1 Answers