1
votes

I have a Pipeline job in Azure Data Factory which I want to use to run the pipeline job but pass all files for a specific month through for example.

I have a folder called 2020/01 inside this folder is numerous files with different names.

The question is: Can one pass a parameter through to only extract and load the files for 2020/01/01 and 2020/01/02 if that makes sense?

2

2 Answers

1
votes

Excellent, Thanks Jay it worked and i can now run my pipeline jobs passing through the month or even day level.

Really appreciate your response, have a fantastic day.

Regards

Rayno

0
votes

The question is: Can one pass a parameter through to only extract and load the files for 2020/01/01 and 2020/01/02 if that makes sense?

You did't mention which connector you are using in pipeline job,but you mentioned folder in your question.As i know,the majority folder path could be parametrization in ADF copy activity configuration.

You could create a param :

enter image description here

Then apply it in the wildcard folder path:

enter image description here

Even if your files' names have same prefix,you could apply 01*.json on the wildcard file name property.