Selecting the current month folder in source using Azure data Factory

1
votes

I want my azure pipeline to always select the current month folder in my source copy activity container and don't want to manually change the folder to the new month every month for example:

enter image description here

You can see the folder points to this month, but when we enter Jan 2021 the folder must automatically point to Jan 2021 and all the contents in that new month folder.

Is there a way to achieve this using dynamic content?

I have tried this string but it does not seem to work:

@concat(pipeline().parameters.ExecutionFolder, '/',formatDateTime(utcnow(), 'yyyy'), '/',formatDateTime(utcnow(), 'MM')

Regards

1
azureazure-data-factoryazure-data-factory-pipeline

1 Answers

1
votes

I've created a test and it works well. I'm using Azure data lake gen2 as file storage.
This is my file structure:
enter image description here

  1. I've created a source dataset to link the container. enter image description here

  2. Declare a parameter with the default value stderr. enter image description here

  3. At the source setting. I select Wildcard file path option and enter @concat(pipeline().parameters.ExecutionFolder, '/',formatDateTime(utcnow(), 'yyyy'), '/',formatDateTime(utcnow(), 'MM')). The end of the path is a wildcard *. enter image description here

  4. After I run debug, all files in this path have been copied. enter image description here