Custom LogBack appender fails with java.lang.ClassNotFoundException

1
votes

I have a custom LogBack appender. The class that uses the appender and the custom appender are compiled into a single jar file. The application is run on DSE spark cluster. I have made the logback.xml changes on each node. When I run the application I see error 

17:39:35,039 |-ERROR in ch.qos.logback.core.joran.action.AppenderAction - Could not create an Appender of type [com.test.logger.CassandraAppender]. ch.qos.logback.core.util.DynamicClassLoadingException: Failed to instantiate type com.test.logger.CassandraAppender

Caused by: java.lang.ClassNotFoundException: com.test.logger.CassandraAppender
        at      at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at      at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at      at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
        at      at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at      at ch.qos.logback.core.util.OptionHelper.instantiateByClassNameAndParameter(OptionH
elper.java:60)

The jar file is copied to /home/[myuser] directory and executed from there. What is missing? Where does Logback look for custom appender?

1
javalogbackappender
Is the jar on your classpath? If not add it to classpath. If using eclipse IDE add it to lib folder in IDE for ease. - Vinay Prajapati
Both my spark job which is using the custom appender, and custom appender are in the same project so they are compiled into a single jar file. So not sure how this will help. - vishakha
Take a look at OptionHelper.instantiateByClassNameAndHelper at line 60. Are you seeing any error in IDE there? I'm just try to get more info from you. - Vinay Prajapati
Caused by: java.lang.ClassNotFoundException: com.test.logger.CassandraAppender at at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at at ch.qos.logback.core.util.OptionHelper.instantiateByClassNameAndParameter(OptionH elper.java:60) - vishakha
let me know if you need more details. - vishakha

1 Answers

0
votes

It seems that the way Logback loads Appenders doesn't play nice with Spark (I ran into this issue with logstash-logback-encoder). I had success with putting the Jar containing the Appender class into the ${SPARK_HOME}/jars directory.