Communication with HDFS: Exception in thread "main" java.io.IOException: Failed on local exception: java.io.EOFException

2
votes
public static void main(String[] args) throws IOException{
    Configuration conf = new Configuration();
    conf.addResource(new Path("/home/myname/hadoop-1.2.1/conf/core-site.xml"));
    conf.addResource(new Path("/home/myname/hadoop-1.2.1/conf/hdfs-site.xml"));
    System.out.println("Attempting initialization of FileSystem");
    FileSystem fs = FileSystem.get(conf);
    System.out.println("Initialization done");
    System.out.println(fs.getHomeDirectory());
    System.out.println(fs.getWorkingDirectory());
    fs.mkdirs(new Path("abcd"));
   }

The syso on the Home Directory and Working directory are working fine and pointing to correct locations on the HDFS. But any form of I/O like reading a file, copy from local, mkdirs is not working. 

    Exception in thread "main" java.io.IOException: Failed on local exception: java.io.EOFException; Host Details : local host is: "localhost/127.0.0.1"; destination host is: ""localhost":9000; 
    at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:738)
    at org.apache.hadoop.ipc.Client.call(Client.java:1099)
    at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:195)
    at com.sun.proxy.$Proxy6.mkdirs(Unknown Source)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:102)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:67)
    at com.sun.proxy.$Proxy6.mkdirs(Unknown Source)
    at org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:1732)
    at org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:1703)
    at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:479)
    at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1807)
    at com.company.trial.DummyClass.main(DummyClass.java:25)
Caused by: java.io.EOFException
    at java.io.DataInputStream.readInt(DataInputStream.java:392)
    at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:823)
    at org.apache.hadoop.ipc.Client$Connection.run(Client.java:757)

The Namenode/Datanode/JobTracker/TaskTracker are working just fine.

I have read here and there that this might be a jar mismatch issue. Jars being used :
Hadoop-common : 0.23.11 Haddop-hdfs : 0.23.11 hadoop-code : 1.2.1

My Hadoop version is 1.2.1.

core-site.xml 

<property>
        <name>fs.default.name</name>
        <value>hdfs://localhost:9000</value>
    </property>

hdfs-site.xml

<property>
       <name>dfs.replication</name>
       <value>1</value>
    </property>
    <property>
       <name>dfs.name.dir</name>
       <value>~/hadoop/hdfs</value>
    </property>

If this is a jar issue, kindly tell me which jars (common and hdfs) is compatible with hadoop-1.2.1

1
hadoop

1 Answers

0
votes

Just add the following code to your API, after the conf sets you already have:

conf.set("fs.default.name", "hdfs://localhost:9000");

Check it, it should work.

Ensure that the following jars added to the Library:

hadoop-core-1.2.1.jar
commons-cli-1.2.jar
commons-logging-1.1.1.jar
commons-lang-2.4.jar
commons-configuration-1.6.jar