Convert a CSV file to JSON using Apache NiFi

1
votes

I am trying to read a csv from local file system and convert the content into JSON format using Apache Nifi and put the JSON format file in the local system. I have succeeded in converting the first row of csv file but not other rows. What am I missing?

Input: 1,aaa,loc1 2,bbb,loc2 3,ccc,loc3

and my nifi workflow is as here: http://www.filedropper.com/mycsvtojson

My output is as below which is desired format but I want that to happen for all the rows.

{ "id" : "1", "name" : "aaa", "location" : "loc1" }

2
apache-nifi

2 Answers

7
votes

There are a few different ways this could be done...

  • A custom Java processor that reads in a CSV and converts to JSON
  • Using the ExecuteScript processor to do something similar in a Groovy/Jython script
  • Use SplitText to split your original CSV into single lines, then use your current approach with ExtractText and ReplaceText, and then a MergeContent to merge back together
  • Use ConvertCsvToAvro and then ConvertAvroToJson

Although the last option makes an extra conversion to Avro, it might be the easiest solution requiring almost no work.

2
votes

This question is a bit older, but there is now a ConvertRecord processor in NiFi 1.3 and newer, which should be able to handle this conversion directly for you, and it avoids having to use split up the data by creating a single JSON array with all of the values, if that is desirable.