Azure data factory V2 copy data issue - error code: 2200 An item with the same key has already been added

Question

I am trying to copy data from a csv file into the Azure sql but I am getting an unique error only during the deployment of pipeline. I am using a normal copy data

{ "errorCode": "2200", "message": "ErrorCode=InvalidParameter,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The value of the property '' is invalid: 'An item with the same key has already been added.'.,Source=,''Type=System.ArgumentException,Message=An item with the same key has already been added.,Source=mscorlib,'", "failureType": "UserError", "target": "Copy data1", "details": [] }

Kindly help to solve

Can you show us your structure of table in your sql server and a sample csv file like yours? — Steve Zhao
I am facing the same issue, while copying a csv file to parquet using a copy activity. — hagarwal

hagarwal hagarwal · Accepted Answer · 2020-10-22T03:28:36

The above error is due to a duplicate column/header in the csv file. The exception message states An item with the same key has already been added that is the duplicate column might be getting added to a dictionary and the exception has been thrown.

Azure data factory V2 copy data issue - error code: 2200 An item with the same key has already been added

1 Answers