Getting error while using custom activities in an Azure Data Factory pipeline

Question

I am getting an error when running a custom activity in Azure Data Factory. The error is:

Job encountered scheduling error. Code: BlobDownloadMiscError Category: ServerError Message: Miscellaneous error encountered while downloading one of the specified Azure Blob(s).

I have been trying to follow the steps in this document https://azure.microsoft.com/en-in/documentation/articles/data-factory-use-custom-activities/

Answer 1

You need a general purpose Azure Storage Account for the Azure Batch to successfully download the assembly to Pool nodes. You got the error possibly because you use Blog Storage only storage account.

Answer 2

The error is thrown by your Azure Batch Pool and from the message it seems your VM could have transient connectivity issue so I would suggest either retry the failed slice to see if that goes away. If not, then I would try reboot the Pool VMs or even switch to a new Pool.

Thanks, Yitao

Answer 3

The problem is that you deployed a custom pipeline in a zipped file and in the meantime the container name containing that pipeline was renamed. The specified Azure Blob is the pipeline you deployed.