I have a program in Apache Flink. I tested and ran it on the local machine and every thing works fine. To run the program on a remote cluster, I did necessary changes as mentioned in Apache Flink Official Website.
I did the following changes:
The two points below
ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();ExecutionEnvironment env = ExecutionEnvironment.createRemoteEnvironment("taskManagerName",
portNo,paralelismNo);
Fixing the necessary paths to read input files and write outputs.
Generate a thin jar out of the program and put the necessary jar libraries into a folder besides my project jar file called
myproj.jar.copying the data and the jar library and myproj.jar into the cluster and run the following command remotely on the cluster:
java -cp pathToJarLib \\* -jar myproj.jar
But, I get the below error and I don't have any clue to fix the issue. There are no relevant log files which can aid me in fixing this issue.
Error:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flink/api/common/functions/MapFunction at java.lang.Class.getDeclaredMethods0(Native Method) at java.lang.Class.privateGetDeclaredMethods(Class.java:2570) at java.lang.Class.getMethod0(Class.java:2813) at java.lang.Class.getMethod(Class.java:1663) at sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494) at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486) Caused by: java.lang.ClassNotFoundException: org.apache.flink.api.common.functions.MapFunction at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ... 6 more
