NiFi header index out of range

Question

I'm converting pipe delimited text file to parquet. to do so I'm using convert record processor.

Reader-csv reader

writer-ParquetSetWriter.

schema strategy- InferSchema in reader and inherit record schema in writer.

But I'm getting an error in ConvertRecord processor. It says that index for header "ColumnName" is 24 but has only 24 values.

enter image description here

Helps to share an example of your header & a row of data. Just from the error I'd assume the index is starting at 0 so value 24=index 23 - but need more info — Sdairs

Mike Thomsen Mike Thomsen · Accepted Answer · 2020-12-08T13:42:16

Based on the input provided, it looks like you have a line that has 25 values because of an improperly delimited set of values. "On index 24, but there are only 24 values" would mean you are on position 25 from an offset of 0.

To debug this, if it's a really big CSV file, you can chain together SplitRecord and ValidateRecord to try to catch the line that has this problem.

NiFi header index out of range

1 Answers