Hadoop not using config files?

Question

I'm deploying Hadoop at work and I've been troubleshooting some days. Yesterday it was working perfectly but today something strange is happening.

I have hadoop.tmp.dir set in core-site.xml as well as other directories for HDFS (datanode, namenode and secondarynamenode in hdfs-site.xml). But today, when I format the FS it's creating all the files in /tmp and not in /usr/local/hadoop/tmp which is the one I have configured.

$ bin/hdfs namenode -format
[...]    
INFO common.Storage: Storage directory /tmp/hadoop-hadoop/dfs/name has been successfully formatted.
[...]

core-site.xml

<property>
      <name>hadoop.tmp.dir</name>                
      <value>/usr/local/hadoop/tmp</value>
</property>

hdfs-site.xml

   <property>
        <name>dfs.namenode.name.dir</name>
        <value>file:/usr/local/hadoop/hdfs/namenode</value>
   </property>

   <property>
        <name>dfs.namenode.checkpoint.dir</name>
        <value>/usr/local/hadoop/hdfs/secondname</value>
   </property>

   <property>
        <name>dfs.datanode.data.dir</name>
        <value>/usr/local/hadoop/hdfs/datanode</value>
   </property>

Anyone has any clue about what's happening?

Thanks!

If you're deploying Hadoop, I suggest you use Apache Ambari rather than editing XML yourself. Also, what is your HADOOP_CONF_DIR environment variable set to? — OneCricketeer
It's not set as it uses automatically $HADOOP_HOME/etc/hadoop (which is correct and HADOOP_HOME set correctly) — RbeJG
Are you sure those variables are set for the same user account starting or running the hadoop commands? Because it seems you're still getting the default property values — OneCricketeer

Shiva Kumar SS Shiva Kumar SS · Accepted Answer · 2018-07-13T11:03:33

make sure this directory exists and have enough permission

give the path as file:///usr/local/hadoop/tmp

Hadoop not using config files?

2 Answers