Copying files (using wildcard) and add timestamp to the file names

0
votes

I have a copy data activity in ADF that copies files using wildcard paths (*.csv -> 20210102_f1.csv, 20210102_f2.csv) into Sink dataset. When it copies the files, I would like them to have a timestamp yyyyMMddhhmmss before the extension name e.g. 20210102_f1_20210202101521.csv.

Is this possible in ADF - copying with wildcards and adding timestamp to target files (all at once, not doing foreach for each of the file and affixing the timestamp)?

1
azure-data-factoryazure-data-factory-2azure-data-factory-pipeline
Do you know what the filenames will contain upfront, or can it be anything? – Ruben Ravnå
it can be anything – kayeesp
ideally, it can be anything. but if there's no other way, im willing to explore specific filenames – kayeesp

1 Answers

0
votes

If you know the filename, you can use this in Sink dataset filename:

@concat('20210102_f1_',formatDatetime(utcnow(),'dd-MM-yyy'),'.csv')

If you still want to loop through every file in folder you can use solution from this thread:

Azure ADF V2 ForEach File CopyData from Blob Storage to SQL Table

Basically you need to get filenames into data factory variables, to use source filename in this dynamic destination filename solution.