NiFI custom processor ClassNotFoundException

Question

I am trying to load a custom NiFi processor, but cannot get NiFi to load all my .nar dependencies, despite trying various pom.xml configurations. I've come across some similar questions on SO but have not found the answer to this issue.

I package my processor as a .nar with mvn clean install, then copy the .nar to NIFI_HOME/lib. Currently I get java.lang.ClassNotFoundException: org.apache.http.conn.HttpClientConnectionManager. Current pom dependencies:

<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <maven.compiler.source>1.7</maven.compiler.source>
    <maven.compiler.target>1.7</maven.compiler.target>
    <nifi.version>1.1.2</nifi.version>
</properties>
<dependencies>
    <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-api</artifactId>
        <version>${nifi.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-utils</artifactId>
        <version>${nifi.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-standard-services-api-nar</artifactId>
        <version>${nifi.version}</version>
        <type>nar</type>
    </dependency>
    <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-ssl-context-service</artifactId>
        <version>${nifi.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-ssl-context-service-api</artifactId>
        <version>${nifi.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-processor-utils</artifactId>
        <version>${nifi.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.httpcomponents</groupId>
        <artifactId>httpclient</artifactId>
        <version>4.4.1</version>
    </dependency>
    <dependency>
        <groupId>org.apache.httpcomponents</groupId>
        <artifactId>httpcore</artifactId>
        <version>4.4.1</version>
    </dependency>
    <dependency>
        <groupId>joda-time</groupId>
        <artifactId>joda-time</artifactId>
        <version>2.9.7</version>
    </dependency>
</dependencies>

My nar's NIFI_HOME/.../bundled-dependencies/ contains all specified jars:

bcpkix-jdk15on-1.55.jar                joda-time-2.9.7.jar
bcprov-jdk15on-1.55.jar                nifi-api-1.1.2.jar
commons-codec-1.9.jar                  nifi-processor-utils-1.1.2.jar
commons-io-2.5.jar                     nifi-security-utils-1.1.2.jar
commons-lang3-3.4.jar                  nifi-ssl-context-service-1.1.2.jar
commons-logging-1.2.jar                nifi-ssl-context-service-api-1.1.2.jar
httpclient-4.4.1.jar                   nifi-utils-1.1.2.jar
httpcore-4.4.1.jar

Full stack trace:

2017-03-13 11:11:23,146 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.alar.nifi.processors.GetIBright could not be instantiated
java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.alar.nifi.processors.GetIBright could not be instantiated
    at java.util.ServiceLoader.fail(ServiceLoader.java:232) ~[na:1.8.0_111]
    at java.util.ServiceLoader.access$100(ServiceLoader.java:185) ~[na:1.8.0_111]
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384) ~[na:1.8.0_111]
    at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) ~[na:1.8.0_111]
    at java.util.ServiceLoader$1.next(ServiceLoader.java:480) ~[na:1.8.0_111]
    at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:116) ~[nifi-nar-utils-1.1.2.jar:1.1.2]
    at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:97) ~[nifi-nar-utils-1.1.2.jar:1.1.2]
    at org.apache.nifi.NiFi.<init>(NiFi.java:139) ~[nifi-runtime-1.1.2.jar:1.1.2]
    at org.apache.nifi.NiFi.main(NiFi.java:262) ~[nifi-runtime-1.1.2.jar:1.1.2]
Caused by: java.lang.NoClassDefFoundError: org/apache/http/conn/HttpClientConnectionManager
    at java.lang.Class.getDeclaredConstructors0(Native Method) ~[na:1.8.0_111]
    at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671) ~[na:1.8.0_111]
    at java.lang.Class.getConstructor0(Class.java:3075) ~[na:1.8.0_111]
    at java.lang.Class.newInstance(Class.java:412) ~[na:1.8.0_111]
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) ~[na:1.8.0_111]
    ... 6 common frames omitted
Caused by: java.lang.ClassNotFoundException: org.apache.http.conn.HttpClientConnectionManager
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[na:1.8.0_111]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[na:1.8.0_111]
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) ~[na:1.8.0_111]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[na:1.8.0_111]
    ... 11 common frames omitted

Can you provide the stacktrace for the class not found exception? — Bryan Bende
@JustinMoore hope you have restarted NiFi server after placing the .jar files under lib folder — Pons
@Pons yep, I have restarted NiFi, the error occurs when it tries to load the new custom processor on startup. Also - it is a .nar file, not .jar — Justin Moore
One thing that is not correct (but should not be related to this problem) is you shouldn't be including the controller service JARs inside your NAR. Your NAR pom should have a NAR dependency on the controller service API, and your processors pom should have a provided dependency on the API so that it is available at compile time but not included at build time. cwiki.apache.org/confluence/display/NIFI/… — Bryan Bende

Justin Moore Justin Moore · Accepted Answer · 2017-03-13T21:03:24

Resolved this by modifying my single pom.xml into separate build (nar) and compile (jar) pom.xml files, with a parent pom.xml. Not sure why it did not work before but I have included the pom.xml files below

parent pom.xml:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <artifactId>nifi-ibright-bundle</artifactId>
    <packaging>pom</packaging>
    <groupId>com.alar.nifi</groupId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>1.7</maven.compiler.source>
        <maven.compiler.target>1.7</maven.compiler.target>
        <nifi.version>1.1.2</nifi.version>
    </properties>

    <modules>
        <module>nifi-ibright-processor</module>
        <module>nifi-ibright-nar</module>
    </modules>
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>com.alar.nifi</groupId>
                <artifactId>nifi-ibright-processor</artifactId>
                <version>1.0-SNAPSHOT</version>
            </dependency>
        </dependencies>
    </dependencyManagement> 
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.nifi</groupId>
                <artifactId>nifi-nar-maven-plugin</artifactId>
                <version>1.0.0-incubating</version>
                <extensions>true</extensions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.15</version>
            </plugin>
        </plugins>
    </build>
</project>

nifi-ibright-nar/pom.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <groupId>com.alar.nifi</groupId>
        <artifactId>nifi-ibright-bundle</artifactId>
        <version>1.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.alar.nifi</groupId>
    <artifactId>nifi-ibright-processor</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>jar</packaging>


    <dependencies>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-api</artifactId>
            <version>${nifi.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-utils</artifactId>
            <version>${nifi.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-ssl-context-service-api</artifactId>
            <version>${nifi.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-processor-utils</artifactId>
            <version>${nifi.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpclient</artifactId>
            <version>4.4.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpcore</artifactId>
            <version>4.4.1</version>
        </dependency>
        <dependency>
            <groupId>joda-time</groupId>
            <artifactId>joda-time</artifactId>
            <version>2.9.7</version>
        </dependency>
    </dependencies>

</project>

NiFI custom processor ClassNotFoundException

1 Answers