SSIS error The column delimiter for column “x” was not found

0
votes

I have a CSV file that looks like below

enter image description here

In my SSIS package (Visual Studio 2017), I create a Flat File connection

Text qualifer: "

Header Row Delimiter: {LF}

Header Rows to skip: 0

Column Delimiter for all columns: Comma {,}, except for the last column WCASH whose Column Delimiter = {LF}

When running the package, I get the error in the Flat File Source "The column delimiter for column "WCASH" was not found." "An error occurred while skipping data rows."

I tried many of the suggestions online (increased WCASH column to something like 3000 characters, delete the Flat file connection and recreate it), but still can't fix the error.

Do you know how I cann fix this error ?

Thank you

1
csvssisdelimiterflat-fileimport-from-csv

1 Answers

0
votes

You have some interesting features in your spreadsheet. The column names in the first row have trailing spaces in them. The columns values in rows 2+ have spaces after the closing double quotes (3rd column), and leading spaces for the fourth and final two columns. This smells like a file generated from a mainframe and forced into a CSV.

Were it me, I would not identify this a column delimited file (based on the 2 lines of data). This is a ragged right file format that someone has added text delimiters and column delimiters to.

  • Use a LF as the header row delimiter
  • header rows to skip = 1
  • uncheck column names in first row

enter image description here

In the Columns tab, set a column where the quotes and commas are.

enter image description here

In the Advanced tab, every other column is named with ignore and the next column will be the actual data.

enter image description here

In my data flow, I pulled in the ignore columns to show that they did contain junk but in your package, you would not want to do this.

enter image description here

What's left?

Cleanup.

You can't convert your numbers to a number data type as those leading spaces will cause issues (if memory serves me correctly). Plus, the string data may have trailing spaces as well so I would have a derived column that applies TRIM to all the columns.

From that point, then try and convert to a strongly typed value and land it in your database.

Here's my sample input file (saved with unix/linefeed/LF endings)

"NAME  ","OPEDT ","OBJ","PCT","MGR   ","rr    ","WCASH "       
"AAAA","07/12/92","BELL"  ,    3.23,"AUJ",    0,       12364.00
"BBBB","01/05/91","PELL"  ,    0.78,"ACC",    0,        9879.00