Azure Data Factory Dataset Dynamic Folder Path

1
votes

I have a data set that resides under a folder path where the date is dynamic (e.g. rootfolder/subfolder/yyyy/mm/dd/subfolder/subfolder), and I am trying to pull it with a copy activity. So far I cannot get Data Factory to recognize that my date is dynamic...

This is the code that I have tried so far:

["rootfolder/subfolder/subfolder/subfolder/subfoler/@{formatDateTime(utcnow(),'yyyy')}/@{formatDateTime(utcnow(),'MM')}/@{formatDateTime(utcnow(),'dd')}/subfolder/file"]
2
azureazure-data-factory
Note the @ symbol is for the start of the expression only, you don't need to repeat it throughout the expression. The curly braces { are for what's called string interpolation, basically when you want to embed this expression within a string. See here for a similar answer recently. - wBob
This can work if you try .../subfoler/{Year}/{Month}{Day}/subfolder/file. - Kiril1512

2 Answers

2
votes

You need to make use of concat function provided by data factory.

@concat('rootfolder/subfolder/subfolder/subfolder/subfolder/',formatDateTime(utcnow(),'yyyy'),'/',formatDateTime(utcnow(),'MM'),'/',formatDateTime(utcnow(),'dd'),'/subfolder/file')

The concat function is similar as in programming languages which concats the strings.

More details: Azure Data Factory Loop Through Files

1
votes

Just to build pm Anish K answer you can also shorten this a bit by using formatting 

formatDateTime(utcnow(),'yyyy/MM/dd')

So final answer would be

@concat('rootfolder/subfolder/subfolder/subfolder/subfolder/',formatDateTime(utcnow(),'yyyy//MM/dd'),'/subfolder/file')

In case you want to learn a bit more on parametrization on ADF feel free to check out this video https://youtu.be/pISBgwrdxPM