Error using data factory for copyactivity from blob storage as source

1
votes

Why do I keep getting this error while using a folder from a blob container as source (which contains only one GZ compressed file) in copy activity in data factory v2 and as sink another blob storage (but I want the file decompressed)?

 "message":"ErrorCode=UserErrorFormatIsRequired,
'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,
Message=Format setting is required for file based store(s) in this scenario.,Source=Microsoft.DataTransfer.ClientLibrary,'",

I know it means I need to specify explicitly the format for my sink dataset, but I am not sure how to do that.

3
azureazure-data-factoryazure-data-lakeazure-data-factory-2
is your source a CSV or Parquet ? if you don't care about the format, you can use binary format to do the copy. - Atvoid
I am using as a source a blob container folder where a GZ compressed file is stored (like a stage area) and as a sink another folder from the same blob container where I want my file to be uncompressed when copied. - Mihnea

3 Answers

2
votes

I suggest using the copy data tool.

step 1

copy tool

step 2 binary copy

1
votes

According you comment, I tried a lot times, unless you choose the compressed file as source dataset and import the schemas, Azure Data factory copy actives will not help you decompress the file.

If the files in the the compressed file don't have the same schema, the copy active also could be failed.

Hope this helps.

0
votes

The easiest way to do this: go to the dataset, and click on the tab Schema, then Import Schema.

enter image description here

Hope this helped!!