Hadoop pseudo distributed configuration on CentOS

1
votes

I am able to configure hadoop on stand alone machine and now try to run pseudo distributed hadoop configuration I have done configuration as per this link.

  1. I am able to format the name node
  2. But when I fire bin/start-all.sh command, the nodes are not started properly.
  3. when I went through log file, I found the following error.

namenode-hadoop2.log

java.io.IOException: File /tmp/hadoop-root/mapred/system/jobtracker.info could only be
replicated to 0 nodes, instead of 1 at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)

jobtracker-hadoop2.log

2013-05-10 11:36:09,959 WARN org.apache.hadoop.hdfs.DFSClient: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /tmp/hadoop-root/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1

  1. I execute following command:

    bin/hadoop fs -put conf input

13/05/10 11:42:19 ERROR hdfs.DFSClient: Exception closing file /user/root/input/mapred-site.xml : org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /user/root/input/mapred-site.xml could only be replicated to 0 nodes, instead of 1

Please tell me any other configuration other than this link.

Thanks.

1
hadoop
Can you check the log file for your data node and post back any errors in that file - Chris White

1 Answers

0
votes

This error implies that there is some problem with your datanode. Make sure it is running and is able to talk to namenode. Also make sure you have enough space left on the datanode's disk. Check if the reserved space for a DN (as set in dfs.datanode.du.reserved) is greater than the remaining free space. You may also get this message due to permissions, eg if JT can not create jobtracker.info on startup.

You may find this link useful. I have tried to show the complete procedure to setup a pseudo distributed cluster. See if it help.