Running hadoop jar in Amazon EMR

Question

I created a custom jar for a map-reduce app and tried running it in Amazon EMR job flow. I compiled the code in hadoop 1.0.4, however Amazon EMR has support for Hadoop 1.0.3. Also I compiled the code under jdk 1.7, and I am not sure which version of java does Amazon EMR use and where can we change that ?

Here is the stacktrace

Exception in thread "main" java.lang.UnsupportedClassVersionError: com/test/GWASMapReduce : Unsupported major.minor version 51.0
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.apache.hadoop.util.RunJar.main(RunJar.java:180)

Muel Muel · Accepted Answer · 2012-11-07T23:22:56

It appears that Java 7 isn't supported by Amazon EMR. You'll need to try compiling your jar to to target an older JVM:

javac -target 1.6 -cp .... etc

Running hadoop jar in Amazon EMR

4 Answers