Why does running the example flink app throw this error?

Question

I'm trying to run the demo app described in https://ci.apache.org/projects/flink/flink-docs-release-1.10/dev/projectsetup/java_api_quickstart.html.

I generated the project with the given script:

curl https://flink.apache.org/q/quickstart.sh | bash -s 1.10.0

and built with:

mvn clean package

But when I try running with:

java -jar targets/java -jar quickstart-0.1.jar

I get the following errors:

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flink/streaming/api/environment/StreamExecutionEnvironment
        at org.myorg.quickstart.StreamingJob.main(StreamingJob.java:39)
Caused by: java.lang.ClassNotFoundException: org.apache.flink.streaming.api.environment.StreamExecutionEnvironment
        at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
        ... 1 more

I have flink in another folder, and have ran bin/start_cluster.sh. I have also been able to run the prepackaged flink applications (in /bin) without issues.

Till Rohrmann Till Rohrmann · Accepted Answer · 2020-04-01T09:55:32

Per default, the pom.xml generated by the quickstarts script will not include Flink's dependencies which are required to run the jar directly via java -jar .... In order to run the jar this way, you would need to build an uber-jar which includes all required Flink dependencies. You can do this via activating the profile add-dependencies-for-IDEA:

mvn clean package -Padd-dependencies-for-IDEA

or by changing the scope of the Flink dependencies from provided to compile.

The reason why Flink's dependencies are not included by default is because when you submit a job to a Flink cluster, then this cluster already contains these dependencies. Therefore, one can keep the size of the generated user jar small by setting the dependencies to provided in the pom.xml, which is equivalent to not including them in the jar.

Why does running the example flink app throw this error?

1 Answers