Issue when flink upload a job with stream sql query

Question

I used the latest flink version(1.10.0) and sbt(1.3.7). I have this exception when upload a job with streaming sql query:

Caused by: java.lang.ClassCastException: org.codehaus.janino.CompilerFactory cannot be cast to org.codehaus.commons.compiler.ICompilerFactory at org.codehaus.commons.compiler.CompilerFactoryFactory.getCompilerFactory(CompilerFactoryFactory.java:129) at org.codehaus.commons.compiler.CompilerFactoryFactory.getDefaultCompilerFactory(CompilerFactoryFactory.java:79) at org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.compile(JaninoRelMetadataProvider.java:432)

When I running main class with sbt run it works perfectly. I made jar with the sbt assembly command and I have conflicts between libraries. For this reason add this in the build.sbt:

assemblyMergeStrategy in assembly := {
     case PathList("META-INF", xs @ _*) => MergeStrategy.discard
     case x => MergeStrategy.first
    }

I read a similar case with hive connector https://issues.apache.org/jira/browse/FLINK-14849 and this is the answer:

After https://issues.apache.org/jira/browse/FLINK-13749 , flink-client will use default child-first resolve-order. If user jar has some conflict dependents, there will be some problem.

My question is: How to resolve these conflicts? Any assembly merge strategy suggest for this case? Help would be appreciated.

Try to exclude janine from your non-flink dependencies or at least exclude it from the fat jar. — Arvid Heise

leilinee leilinee · Accepted Answer · 2020-05-21T06:38:27

I have same problem. Maybe you have package your code with flink-table-planner_${scala.binary.version}, so you need to change your maven config with those settings:

    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-table-planner_${scala.binary.version}</artifactId>
        <version>${flink.version}</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-table-planner-blink_${scala.binary.version}</artifactId>
        <version>${flink.version}</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-table-planner_${scala.binary.version}</artifactId>
        <version>${flink.version}</version>
        <scope>provided</scope>
    </dependency>

Issue when flink upload a job with stream sql query

1 Answers