0
votes

I am creating a test Azure Data Factory Pipeline for learning purposes.

For this specific pipeline, I want to move files from one blob to the other. In ADF world, this involves creating a pipeline with the following activities:

  1. Get Metadata: retrieve file list from Blob 1
  2. Filter: from the output of previous activity, filter out folders, and output list of files
  3. Move files: get output from previous activity, move files from Blob 1 to Blob 2

ADF does not implement a Move File activity, so activity 2. above is actually a ForEach activity composed by two sub-activities:

2.1 Copy Files: Copy file from Blob 1 to Blob 2 2.2 Delete Files:Delete file from Blob 1

When debugging this pipeline, I am getting the following error for activity 2.2:

Failed to execute delete activity with data source 'AzureBlobStorage' and error 'The required Blob is missing. Folder path: sensor-sink-aws/test - Copy.json/.'

But the file exists in the container:

enter image description here

In my DeleteFile activity, I have the following data for Source:

enter image description here

Which basically references the Source Dataset that I created for this pipeline - it is a reusable dataset, meaning that I'm able to pass the container name and file name in a dynamic way.

Any idea on what could be going wrong?

1

1 Answers

0
votes

From the path in the error it looks that the path of the blob is not correct "sensor-sink-aws/test - Copy.json/.'"

The file name should had been at the last , but as you can see that it "/." . I think you should check Delete activity -> Source ->Dataset -> Open-> Connection -> It should be some like the below screenshot ( please pay attention that the file name is at the end , my guess is you have put that in the middle text box )

enter image description here