Apache NiFi: FlowFileHandlingException when transfer FlowFile in custom processor

Question

I doing data ingestion from remote API service for different time ranges in my custom NiFi Processor.

I have time ranges global counter which updates with each iteration (I'm using Timer driven scheduling strategy).

When the counter is greater than the maximum value, I want to transfer just FlowFile from request (session.get()) with SUCCESS relationship, i.e. without performing additional logic:

session.transfer(requestFlowFile, SUCCESS);

I undestand that I can't stop or pause processor when time ranges collection is over. So I trying to use the above approach as a solution.

All iterations going fine until the counter has become greater than the maximum and processor trying to transfer FlowFile from request (session.get())

So I having this Exception:

failed to process session due to org.apache.nifi.processor.exception.FlowFileHandlingException: StandardFlowFileRecord[uuid=459e615b-0ff5-424f-aac7-f95d364cdc13,claim=,offset=0,name=99628180019265,size=0] is not known in this session

What's wrong here? Or may be another approach?

Answer 1

That error means the flow file being passed to session.transfer() came from a different session. You can only call transfer() on the same session from which you called get().

Answer 2

if it's a custom processor - just do not do session.get() and skip this execution without transferring anything.

or if you need incoming file to take a decision you can get it, do some checks, and rollback the current session with penalize rollback(true), so the file(s) you get will stay in the incoming queue during Penalty Duration processor parameter without thiggering rocessor to run.

or you can do session.get(FlowFileFilter) to get from incoming queue only files that matches your logic