Hadoop\HDFS: "no such file or directory"

6
votes

I have installed Hadoop 2.2 on a single machine using this tutorial: http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/

Some details were changed a little bit - for example, I used java 8, /hadoop root dir etc. Users, SSH, config keys - the same.

Namenode was successfully formatted:

13/12/22 05:42:31 INFO common.Storage: Storage directory /hadoop/tmp/dfs/name has been successfully formatted.
13/12/22 05:42:31 INFO namenode.FSImage: Saving image file /hadoop/tmp/dfs/name/current/fsimage.ckpt_0000000000000000000 using no compression
13/12/22 05:42:32 INFO namenode.FSImage: Image file /hadoop/tmp/dfs/name/current/fsimage.ckpt_0000000000000000000 of size 198 bytes saved in 0 seconds.
13/12/22 05:42:32 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0
13/12/22 05:42:32 INFO util.ExitUtil: Exiting with status 0
13/12/22 05:42:32 INFO namenode.NameNode: SHUTDOWN_MSG:

However, not 'mkdir' neither even 'ls' command worked:

$ /hadoop/hadoop/bin/hadoop fs -ls
Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library /hadoop/hadoop-2.2.0/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
13/12/22 05:39:33 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
ls: `.': No such file or directory

Thanks for any help guys.

4
hadoophdfs
update: /hadoop/hadoop/bin/hadoop fsck -move prints The filesystem under path '/' is HEALTHY - Spaceman
ls: '.': No such file or directory could mean you have nothing (no file nor folder) in hdfs. You mentioned hadoop fs -mkdir doesn't work as well, what's the error? have you tried hadoop fs -mkdir /tmp? - zhutoulala
@zhutoulala too late =) I switched to Disco MR framework and I'm quite happy with it... Hadoop feels like one of the most user-unfriendly tools I've ever used. "Java style", yeah. - Spaceman
You may take a look at more user-friendly Hadoop distros such as Cloudera or hortonworks, which have everything integrated into a web portal... - zhutoulala
@zhutoulala well, I see no real reasons to migrate - it's just a tool, and I use the one which works fine for me. But thanks for Hortonworks, their website is much better than Cloudera's one, will try their distro some day. - Spaceman

4 Answers

12
votes

In Hadoop 2.4

hdfs dfs -mkdir /input
hdfs dfs -ls /
11
votes

Try 

hadoop fs -ls /

Tested on hadoop 2.4

2
votes

Worked in my case: First Get hadoop installed path by : 

echo ${HADOOP_INSTALL} //in my case output is : `/user/local/hadoop`

Then create directory at your hadoop installed path, If you know your hadoop installed directory ignore above command

hadoop fs -mkdir -p /user/local/hadoop/your_directory

Here hadoop is directory

Tested on hadoop 2.4

2
votes

I have verified this worked in Hadoop 2.5 

hdfs dfs -mkdir /input

(where /input is the HDFS directory)