Apache Nifi : I want to ingest my Data CSV to Elasticsearch without streaming it to some other processor using apache nifi

Question

I am trying to setup a simple process to modify my CSV file and ingest it to the elasticsearch DB using Apache Nifi. I don't want to stream my CSV file on Stdout, while passing my file from one processor to another.

I've already made two flows.

Myfirst flow get my CSV file using GetFile processor, customizes it using ExecuteStreamCommand in which i run my python script to read,modify and save my CSV file locally.

My second flow again read that modified CSV file using GetFile processor and ingest it directly to my ElasticSearch DB.

Now, for getting this task accomplished, i run these two flows seperatly.

Can i connect the ExecuteStreamCommand of my first flow and GetFile of my second flow together, so that i can run them together as one single flow.

Is there any other option to read a file written by ExecuteStreamCommand locally without streaming ??

Answer 1

The output of the command executed by ExecuteStreamCommand will be written to a flow file that is transferred to the "output stream" relationship. You should be able to connect ExecuteStreamCommand "output stream" directly to PutElasticSearch.