Azure data factory On premises file skip header row

Question

Currently I am prototyping an Azure data factory setup where my input is a "on premises file", however when the Copy Activity runs, the header row from the file gets copied to the sink SQL server table.

Is there any way to skip the header row when performing Copy Activity?

Any help would be appreciated.

Answer 1

the way I can think of this is to execute a stored procedure on the destination i.e. (SQL Server sink) in the copy activity. The stored procedure will skip the header row and act on the remaining data.

Thanks, Harish

Answer 2

If you are copying data from a Blob store and you want to copy it to an Azure DB table, you can always modify the pipeline to include skipHeaderLineCount in the properties section, like this

  "properties": {
    "activities": [
        {
            "type": "Copy",
            "typeProperties": {
                "source": {
                    "type": "BlobSource",
                    "skipHeaderLineCount": 1
                },
                "sink": {
                    "type": "SqlSink",
                    "writeBatchSize": 0,
                    "writeBatchTimeout": "00:00:00"
                }
            },