Getting E0902: Exception occured:[Server IPC version 9 cannot communicate with client version 4]

Question

my hadoop version is 2.7.1 and my oozie version is 3.3.2.i have created oozie.war file successfully.But,when i try to create shared library in hdfs using the following command,

oozie-setup.sh sharelib create -fs hdfs://training-21:54310

i get the following exception,

Stack trace for the error was (for debug purposes):
--------------------------------------
org.apache.oozie.service.HadoopAccessorException: E0902: Exception occured: [Server IPC version 9 cannot communicate with client version 4]
    at org.apache.oozie.service.HadoopAccessorService.createFileSystem(HadoopAccessorService.java:433)
    at org.apache.oozie.tools.OozieSharelibCLI.run(OozieSharelibCLI.java:144)
    at org.apache.oozie.tools.OozieSharelibCLI.main(OozieSharelibCLI.java:52)
Caused by: org.apache.hadoop.ipc.RemoteException: Server IPC version 9 cannot communicate with client version 4
    at org.apache.hadoop.ipc.Client.call(Client.java:1107)
    at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
    at com.sun.proxy.$Proxy3.getProtocolVersion(Unknown Source)
    at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:411)
    at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:135)
    at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:276)
    at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:241)
    at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:100)
    at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1411)
    at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
    at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1429)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:254)
    at org.apache.oozie.service.HadoopAccessorService$2.run(HadoopAccessorService.java:425)
    at org.apache.oozie.service.HadoopAccessorService$2.run(HadoopAccessorService.java:423)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1136)
    at org.apache.oozie.service.HadoopAccessorService.createFileSystem(HadoopAccessorService.java:423)
    ... 2 more
-------------------

I modified only the following conf file:

core-site.xml(hadoop)

<configuration>
 <property>
    <name>hadoop.tmp.dir</name>
    <value>/home/user1/tmp</value>
</property>

     <property>
         <name>fs.default.name</name>
         <value>hdfs://TRAINING-21:54310</value>
     </property>
   <property>
     <name>hadoop.proxyuser.user1.hosts</name>
     <value>*</value>
   </property>

   <property>
     <name>hadoop.proxyuser.user1.groups</name>
     <value>*</value>
   </property>
</configuration>

I have tried with different version of oozie(4.2.0) but the problem persists.what can i do to solve this exception?

UST UST · Accepted Answer · 2016-05-08T06:15:12

By default Oozie builds against Hadoop version 0.23.5 so to build against Hadoop 2.7.1, we will have to configure maven dependencies in pom.xml

In the downloaded Oozie source code (pom.xml), the hadoop‑2 maven profile specifies hadoop.version & hadoop.auth.version to be 2.3.0. So we need to change them to use 2.7.1.

Modify the hadoop-2 profile as follows

<profile>
        <id>hadoop-2</id>
        <activation>
            <activeByDefault>false</activeByDefault>
        </activation>
        <properties>
           <hadoop.version>2.7.1</hadoop.version>
           <hadoop.auth.version>2.7.1</hadoop.auth.version>
           <pig.classifier>h2</pig.classifier>
           <sqoop.classifier>hadoop200</sqoop.classifier>
        </properties>
    </profile>

Go to hadooplibs

cd hadooplibs

Go to hadoop-2

cd hadoop-2

Edit pom.xml as follows

<parent>
    <groupId>org.apache.oozie</groupId>
    <artifactId>oozie-main</artifactId>
    <version>4.1.0</version>
    <relativePath>../../pom.xml</relativePath>
</parent>
<groupId>org.apache.oozie</groupId>
<artifactId>oozie-hadoop</artifactId>
<version>2.7.1.oozie-4.1.0</version>
<description>Apache Oozie Hadoop ${project.version}</description>
<name>Apache Oozie Hadoop ${project.version}</name>
<packaging>jar</packaging>

<dependencies>
    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-client</artifactId>
        <version>2.7.1</version>
        <scope>compile</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-auth</artifactId>
        <version>2.7.1</version>
    </dependency>

Go to hadoop-distcp-2

cd ../hadoop-distcp-2/

Edit pom.xml as follows.

<artifactId>oozie-hadoop-distcp</artifactId>
<version>2.7.1.oozie-4.1.0</version>
<description>Apache Oozie Hadoop Distcp ${project.version}</description>
<name>Apache Oozie Hadoop Distcp ${project.version}</name>
<packaging>jar</packaging>

<dependencies>
    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-distcp</artifactId>
        <version>2.7.1</version>
        <scope>compile</scope>
    </dependency>

Go to hadoop-test-2

cd ../hadoop-test-2

Edit the pom.xml as follows.

<artifactId>oozie-hadoop-test</artifactId>
<version>2.7.1.oozie-4.1.0</version>
<description>Apache Oozie Hadoop ${project.version} Test</description>
<name>Apache Oozie Hadoop ${project.version} Test</name>
<packaging>jar</packaging>

<dependencies>
    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-minicluster</artifactId>
        <version>2.7.1</version>
        <scope>compile</scope>
    </dependency>

Build Oozie with hadoop 2.7.1 as follows.

bin/mkdistro.sh ‐P hadoop‐2 ‐DskipTests

And then follow the usual procedures.

Getting E0902: Exception occured:[Server IPC version 9 cannot communicate with client version 4]

1 Answers