How can I keep the Hadoop File system in a physical disk other than the disk of hadoop installation?

1
votes

I am trying to install hadoop in a multinode cluster environment. I have installed ubuntu 15.10 on an SSD. I want to install hadoop 2.6.2 on the SSD and keep my HDFS on a separate SATA hard drive. For this, what steps should I follow?

I have installed hadoop in SSD with the following configurations in hdfs-site.xml. Hence I have set the property dfs.datanode.data.dir as file:///media/coea23/HDFS/hdfs/datanode. But the datanode is not showing while executing jps whereas the namenode is showing which is in the SSD where the hadoop installation has been done.

<property> 
<name>dfs.datanode.data.dir</name> 
<value>file:///media/coea23/HDFS/hdfs/datanode</value> 
<description>DataNode directory</description> 
</property> 

<property> 
<name>dfs.namenode.name.dir</name> 
<value>file:///home/hduser/hdfs/namenode</value> 
<description>NameNode directory for namespace and transaction logs storage.</description> 
</property>

Please give your valuable suggestion. Thanks in advance.

Kamaruddin

1
hadoophdfs

1 Answers

0
votes

you configure this in the config file...

hdfs-site.xml on each DataNode:

<property>
 <name>dfs.datanode.data.dir</name>
 <value>file:///data/1/dfs/dn,file:///data/2/dfs/dn,file:///data/3/dfs/dn,file:///data/4/dfs/dn</value>
</property>

edit

Note:

dfs.data.dir and dfs.name.dir are deprecated; you should use dfs.datanode.data.dir and dfs.namenode.name.dir instead, though dfs.data.dir and dfs.name.dir will still work.