hdfs dfs -mkdir, No such file or directory

27
votes

Hi I am new to hadoop and trying to create directory in hdfs called twitter_data. I have set up my vm on softlayer, installed & started hadoop successfully.

This is the commend I am trying to run:

hdfs dfs -mkdir hdfs://localhost:9000/user/Hadoop/twitter_data

And it keeps returning this error message: 

 /usr/local/hadoop/etc/hadoop/hadoop-env.sh: line 2: ./hadoop-env.sh: Permission denied
16/10/19 19:07:03 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
mkdir: `hdfs://localhost:9000/user/Hadoop/twitter_data': No such file or directory

Why does it say there is no such file and directory? I am ordering it to make directory, shouldn't it just create one? I am guessing it must be the permission issue, but I cant resolve it. Please help me hdfs experts. I have been spending too much time on what seems to be a simple matter.

Thanks in advance.

2
hadoophdfs
Oh, and I want to be able to view this hdfs on my browser. How can I access it? – 2D_
I tried hdfs dfs -mkdir -p /user/Hadoop/twitter_data, and I get this /usr/local/hadoop/etc/hadoop/hadoop-env.sh: line 2: ./hadoop-env.sh: Permission denied 16/10/19 19:25:37 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable mkdir: Call From master.someplace.net/198.23.84.123 to localhost:9000 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused I need to be able to see this on my browser. – 2D_
btw, can anybody explain to me what the difference is between just creating a directory vs. having hdfs://localhost:9000 in front? I saw examples for both cases. and I just assumed hdfs://localhost:9000 makes is accessible via browser. please correct me if I am wrong – 2D_

2 Answers

45
votes

It is because the parent directories do not exist yet either. Try hdfs dfs -mkdir -p /user/Hadoop/twitter_data. The -p flag indicates that all nonexistent directories leading up to the given directory are to be created as well.

As for the question you posed in the comments, simply type into your browser http://<host name of the namenode>:<port number>/.

8
votes

use the below steps command to create the directory:

1) don't run the hadoop and format the namenode:-

$ hadoop namenode -format

2) run hadoop by :-

$ start-all.sh

3)now first make the initial directory then create the another in same directory:

$ hadoop fs -mkdir /user
$ hadoop fs -mkdir /user/Hadoop
$ hadoop fs -mkdir /user/Hadoop/tweeter_data

Follow the above steps to solve the problem.