Nifi Custom processor exception

3
votes

I am Using Nifi 0.4.1 version. Am writing custom code to convert CSV to avro format. I have created class file and able to generate nar file. Placed nar file in lib directory and restarted nifi server.

class file does not have any errors or warnings.

When drag the processor in nifi work area, am not able to see that class which is created for converting csv to avro.

Any help appreciated..

Thanks,

1
apache-nifi
To clarify, when you say not able to see the class, do you mean the processor does not show up in the list of available processors when you try to add a new one to the canvas? - apiri
Yes.. I am not able to see the class in nifi canvas.. - user3656330

1 Answers

7
votes

You can check a few things to diagnose your processor issue:

  1. Provider Configuration File
  2. Processor Class
  3. NAR in lib directory
  4. ExtensionManager entry in nifi-app.log
  5. Errors in nifi-app.log

It sounds like you have already checked some, but please confirm all of them:

Provider Configuration File

Your processor must be referenced in a service configuration file in the same nar file with the Java code. The file will be named org.apache.nifi.processor.Processor, and will contain one fully-qualified class name per line of Processor implementations. It's helpful to look at an example, like this provider config for nifi-solr-processors.

org.apache.nifi.processors.solr.PutSolrContentStream
org.apache.nifi.processors.solr.GetSolr

Processor Class

Make sure your processor class is public, implements org.apache.nifi.processor.Processor, inherits from org.apache.nifi.processor.AbstractProcessor, or from another Processor implementation. You should ensure that the fully-qualified class name matches the provider configuration file above.

public abstract class SolrProcessor extends AbstractProcessor {
...
public class GetSolr extends SolrProcessor {
...
public class PutSolrContentStream extends SolrProcessor {
...

NAR in lib Directory

Your nar file should be placed in the lib directory of your NiFi installation. Make sure the NiFi user will have permissions to it, if that matters in your setup. Sounds like you checked this already.

ExtensionManager Entry in nifi-app.log

After starting NiFi, look in logs/nifi-app.log. The ExtensionManager will output a listing of all of the extensions found and loaded from the lib directory. It looks like this (edited for length):

org.apache.nifi.nar.ExtensionManager Extension Type Mapping to Classloader:
=== ControllerService type || Classloader ===
org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderControllerService || org.apache.nifi.nar.NarClassLoader[.\work\nar\extensions\nifi-aws-nar-0.6.0-SNAPSHOT.nar-unpacked]
org.apache.nifi.distributed.cache.server.DistributedSetCacheServer || org.apache.nifi.nar.NarClassLoader[.\work\nar\extensions\nifi-distributed-cache-services-nar-0.6.0-SNAPSHOT.nar-unpacked]
org.apache.nifi.distributed.cache.client.DistributedSetCacheClientService || org.apache.nifi.nar.NarClassLoader[.\work\nar\extensions\nifi-distributed-cache-services-nar-0.6.0-SNAPSHOT.nar-unpacked]
...
=== End ControllerService types ===
=== Processor type || Classloader ===
org.apache.nifi.processors.standard.GetHTTP || org.apache.nifi.nar.NarClassLoader[.\work\nar\extensions\nifi-standard-nar-0.6.0-SNAPSHOT.nar-unpacked]
org.apache.nifi.processors.script.ExecuteScript || org.apache.nifi.nar.NarClassLoader[.\work\nar\extensions\nifi-scripting-nar-0.6.0-SNAPSHOT.nar-unpacked]
org.apache.nifi.processors.aws.s3.DeleteS3Object || org.apache.nifi.nar.NarClassLoader[.\work\nar\extensions\nifi-aws-nar-0.6.0-SNAPSHOT.nar-unpacked]
org.apache.nifi.processors.standard.ExecuteSQL || org.apache.nifi.nar.NarClassLoader[.\work\nar\extensions\nifi-standard-nar-0.6.0-SNAPSHOT.nar-unpacked]
org.apache.nifi.processors.mongodb.GetMongo || org.apache.nifi.nar.NarClassLoader[.\work\nar\extensions\nifi-mongodb-nar-0.6.0-SNAPSHOT.nar-unpacked]
...

You want your Processor, Controller Service, or Reporting Task to be found in this list.

Errors in nifi-app.log

If found and loaded, there may also be exceptions thrown as your code is exercised, this will also appear in nifi-app.log. For example, if you put the wrong class name in your provider config file, you get this "Provider not found" error:

java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider org.apache.nifi.processors.test.TestProcessor not found
    at java.util.ServiceLoader.fail(ServiceLoader.java:239) ~[na:1.8.0_65]
    at java.util.ServiceLoader.access$300(ServiceLoader.java:185) ~[na:1.8.0_65]
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:372) ~[na:1.8.0_65]
    at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) ~[na:1.8.0_65]
    at java.util.ServiceLoader$1.next(ServiceLoader.java:480) ~[na:1.8.0_65]
    at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:107) ~[nifi-nar-utils-0.6.0-SNAPSHOT.jar:0.6.0-SNAPSHOT]
    at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:88) ~[nifi-nar-utils-0.6.0-SNAPSHOT.jar:0.6.0-SNAPSHOT]
    at org.apache.nifi.NiFi.<init>(NiFi.java:120) ~[nifi-runtime-0.6.0-SNAPSHOT.jar:0.6.0-SNAPSHOT]
    at org.apache.nifi.NiFi.main(NiFi.java:227) ~[nifi-runtime-0.6.0-SNAPSHOT.jar:0.6.0-SNAPSHOT]