Making log files in NiFi

Question

I want to make log files for each processors in NiFi. I use splitText for splitting log files and then processing them after it I have one log message distribute in 5 files. I want to keep this data and write it in one log file for each processor (for example I use this expression fro getting executescript processor${regex:toLower():contains('executescript')}).

1. How can I write this logs in one log file for each processor?
2. Should I use any native NiFi processor or make it by Groovy code?

Is it possible to get flowfile data I used this but processor seems to have bad response:

def flowFile1 = session.create();
def flowFile=session.get();

while(flowFile != null){
    flowFile1 = session.write(flowFile, {outputStream -> def builder = new groovy.json.JsonBuilder(flowFile) 
    outputStream.write(builder.toPrettyString().getBytes(Standar‌​dCharsets.UTF_8)) } as OutputStreamCallback) 
}

flowFile1 = session.putAttribute(flowFile,'filename','ExecuteScriptLog')
session.remove(flowFile);
session.transfer(flowFile1, REL_SUCCESS)

I have WorkFlow ike thi and i wnat to get connection name for example 'executescrip't and make flowfile with this name and input all flowfile data whcih is placed inside this 'executescript' queue and write it in one file created by me (in this case 'executescript')

Answer 1

the logging configuration you can manage through NIFI_HOME/conf/logback.xml file.

you can define here logging files (appenders) and what messages should be logged.

the logback manual: https://logback.qos.ch/manual/index.html

each processor has a classname that you can see on the screen (ex:org.apache.nifi.processors.attributes.UpdateAttribute) - you need this info to configure logger in logback.xml and direct it to appender (file)

also you can define filtering in the logback.xml for each appender so that only messages that matches regexp will be appended into into it.