azure data factory copy activity pick only first 100 files

Question

I would like to copy files from one blob account to another blob using Azure Data Factory . However my source blob has 25k + files (XML). I just want to copy 100 files for my initial testing.

How to achieve it ?

it can be achieved with many actives，and may be complex and not directly . Is that ok? — Leon Yue
Currently , I am using wildcard so that I dont pick all the files. If something is similar to that it will be helpful. — amit agarwal
Another questions, do the files have same naming rule or named random? — Leon Yue

Pratik Somaiya Pratik Somaiya · Accepted Answer · 2021-01-21T07:23:23

You can use AzCopy to specify selected files to COPY or if you just want to test your pipeline then create another container, place only few files and then point your ADF to new container to test your pipeline.

More information can be found here if you need to use AzCopy

azure data factory copy activity pick only first 100 files

1 Answers