Hi
I have HDF cluster with 3 Nifi instance which lunches jobs(Hive/Spark) on HDP cluster. Usually nifi writes all information to different repositories available on local machine.
My question is - Does nifi writes any data,provenance information or does spilling on HDP nodes (ex. data nodes in HDP cluster) while accessing HDFS,Hive or Spark services ?
Thanks
Apache NiFi does not use HDFS for any of its internal repositories/data. The only interaction between NiFi and Hadoop services would be through specific processors made to interact with these services, such as PutHDFS, PutHiveQL, etc.
Provenance data can be pushed out of NiFi using the SiteToSiteProvenanceReportingTask and then stored in whatever location is appropriate (HDFS, HBase, etc).
