Azure Blob Storage as Destination in Azure Data Factory Pipeline with Specific File name and Compression

Question

I am trying to copy data from azure sql database to Azure Blob Storage.

Source : Azure SQL Database. Destination : Azure Blob Storage.

"typeProperties": {
            "fileName": "MyFile.csv.zip",
            "folderPath": "myfolderpath/",
            "format": {
                "type": "TextFormat",
                "columnDelimiter": ",",
                "nullValue": "",
                "firstRowAsHeader": false
            },

            "compression": {
                "type": "ZipDeflate",
                "level": "Fastest"
            }
        },

Zip File is created perfectly but it contains auto-generated file name after extract (not MyFile.csv).

When I use the GZip as compression the .gz file contains the file with same file name like MyFile.csv

I read documentation, but cannot find anything.

Anybody has faced same issue? Please advise.

Anqing Zhao Anqing Zhao · Accepted Answer · 2017-09-13T08:36:27

Zip is different from GZip. The former is a compressor with archive, and the latter is a file-based compressor.

The filename you specified is only for the name of result zip file, not related to the inner file's name. Considering the data is extract from a SQL source without original file name, we simply use auto-generated names for it.

One zip file may have several inner files. If you are mentioning the inner file's name can be associated to the given zip file's when there is only one inner file. Yes, it is. But we haven't taken such improvement for now.

To mitigate and fulfill your requirement, you can author two activities. The first one get data from SQL to Blob with specified file name (MyFile.csv) as you want. The second one archive it to a zip file (MyFile.csv.zip).

Azure Blob Storage as Destination in Azure Data Factory Pipeline with Specific File name and Compression

2 Answers