Hadoop Pseudo-Distributed java.net.ConnectException: Connection refused on virtual box

Question

I have installed hadoop on ubuntu which running on virtualbox. When I first installed hadoop I can start hdfs and create directories with no problem.

But after I restart the virtual machine I got "Connection refused" error when try to run ls command on the hdfs. Then I added "Port 9000" in sshd_config base on Hadoop cluster setup - java.net.ConnectException: Connection refused now I am getting "ls: Failed on local exception: com.google.protobuf.InvalidProtocolBufferException: Protocol message tag had invalid wire type.; Host Details : local host is: "hadoop/127.0.1.1"; destination host is: "localhost":9000;"

--------------etc/hadoop/core-site.xml -------------

<configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://localhost:9000</value>
    </property>
</configuration>

--------------etc/hadoop/hdfs-site.xml-----------------

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
</configuration>

Answer 1

Ok. I solved it. I run jps and found out the namenode failed to start. Then when I try to figure out what's wrong I keep looking at "hadoop-xxx-namenode-hadoop.out" which has pretty much no useful information. So I looked at "hadoop-xxx-namenode-hadoop.log" which clearly show the error "org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /tmp/hadoop-xxx/dfs/name is in an inconsistent state: storage directory does not exist or is not accessible."

Did more research I found by default Hadoop will put namenode and datanode file in system temp directory. And when I restart then directory got removed. So I had to do is change "hdfs-site.xml" and add "dfs.namenode.name.dir" and "dfs.namenode.data.dir" properties. Then when I try start hdfs I got "Address already in use". The port I was using was 9000 and after reboot it is binded with "tcp 0 0 *:9000 " so I changed "fs.defaultFS" in core-site.xml to use 9001 instead. Now everything can start now.

Hope this can help.