Shutting Down DataNode - Hadoop installation on Windows 10

Question

I am trying to install Hadoop version 3-1.0 and Java 1.8.0_261 on Windows 10(64 bit). Here are my configurations in hdfs-site.xml

<configuration>
 <property>
  <name>dfs.replication</name>
  <value>1</value>
 </property>
 <property>
  <name>dfs.namenode.name.dir</name>
  <value>file:///C:/hadoop-3.1.0/hadoop-3.1.0/data/namenode</value>
 </property>
 <property>
  <name>dfs.datanode.failed.volumes.tolerated</name>
  <value>0</value>
 </property>
 <property>
  <name>dfs.datanode.data.dir</name>
  <value>file:///C:/hadoop-3.1.0/hadoop-3.1.0/data/datanode</value>
 </property>
</configuration>

Everything is installed properly, but when I run start-dfs.cmd, my namenode starts running, but datanode doesn't. It gives the following error:

ERROR datanode.DataNode: Exception in secureMain org.apache.hadoop.util.DiskChecker$DiskErrorException: Too many failed volumes - current valid volumes: 0, volumes configured: 1, volumes failed: 1, volume failures tolerated: 0

I've tried everything, i.e setting dfs.datanode.failed.volumes.tolerated to 1, removing it entirely, but to no help.

I've searched a lot but couldn't get it fixed. Any help will be greatly appreciated. Thanks.

Yes I did. And I got it working by changing hdfs-site.xml from <property> <name>dfs.datanode.data.dir</name> <value>file:///C:/hadoop-3.1.0/hadoop-3.1.0/data/datanode</value> </property> to <property> <name>dfs.datanode.data.dir</name> <value>datanode</value> </property> — Namal159
That achieves the same result as formatting the datanode volume or clearing the directory. An absolute path is preffered there — OneCricketeer
but formatting namenode didn't help. and I didn't format the datanode tho. — Namal159
You moved the data directory. Same effect when you only have one machine and one disk anyway — OneCricketeer

Sarthak Sahu Sarthak Sahu · Accepted Answer · 2021-04-17T15:17:31

I improvised on your answer and it seems to have worked for me. Make the same change of data node into name node as well.

hdfs-site.xml

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
    <property>
        <name>dfs.namenode.name.dir</name>
        <value>namenode</value>
    </property>
    <property>
        <name>dfs.datanode.data.dir</name>
        <value>datanode</value>
    </property>
</configuration>

It creates the datanode and namenode directories in your sbin folder (default dir of start-all.cmd).

Shutting Down DataNode - Hadoop installation on Windows 10

1 Answers