Maven creating two jars from flink project and only 80mb one works in cluster

Question

So currently my flink project produces two jars when "mvn package" is ran. From what I can tell, based on looking around the internet, the maven shade plugin makes the other jar. Here is a snapshot of the jars that are produced in the project's target folder:

So I can upload the 82.9MB one to the flink cluster and run the job and it works fine, but if I try using the original-mtsas-flink-1.3 jar that's 146.4KB, it will throw an error along the lines of "no main class found".

So my question is two parts: 1.) Why are the two jars being produced. 2.) Is there a way to upload the smaller jar and be able to run a job from it?

Thanks for any help in advance!

Answer 1

1) Maven used to create two jars : one with all the dependencies that are in your pom.xml and needed by your program AND one without these dependencies.

The original is your program without any dependencies. That is why it is way smaller than the other one.

2) You can't run your smaller jar because your program needs the dependencies to work. There are ways to use only this jar by defining in the classpath where your program could find the dependencies. But in definitive, the dependencies have to be retrieve in a way or another.