In NIFI how to convert from CSV to JSON without CSV header

3
votes

In NIFI how to convert from CSV to JSON without CSV header. For each CSV row searate json flow file sholud create and send to next processor. Below is the csv

861359032561480,1,15.480237,190506144035,a
861359032561480,3,16.480237,190506144035,b
861359032561480,2,17.480237,190506144035,c

The expected json is :

{"test":861359032561480,"test2":"1","test3":15.48,"test4":190506144035,"test5":A}

In the above JSON second json value 1 is string, in third json value the value is restricted to 2 digits after the decimal, in fourth json value is uppercased.

So, how can I convert and apply these transformations?

1
apache-nifi

1 Answers

3
votes

Use QueryRecord(to convert field to uppercase) processor and then Split the array of json records.

  • Configure Record Reader(CSVReader)/Writer(JsonRecordSetWriter).

  • In JsonSetWriter keep the matching avro schema like long,string,decimal.

In QueryRecord processor add the new property and keep the sql statement to convert "test5" to UPPER(test5) case, using ApacheCalcite sql parser.

Your sql statement would be something like below..

select test,test2,test3,test4,UPPER(test5)test5 from flowfile

Then use SplitRecord(prefered if json file is big) (or) SplitJson processors to split the array of json records to individual flowfiles.

Flow:

1.QueryRecord //to read csv and write in json format
2.SplitRecord (or) SplitJson //to Split array into individual flowfiles
3.other processors.