1
votes

I need to copy data from SFTP folder and need to dynamically pick only the current date minus 1 day file. I need to load this data to ADLS Gen -1. I'm using Copy activity and have parameterised the File path and File name in Dataset and pass these values from Pipeline.

File path - xyz/test File name - @concat('auto-miles-per-gallon',formatDateTime(addDays(utcnow(),-1),'yyyyMMdd'),'.csv').

I'm able to run the copy activity but its not copying only the specified file (auto-miles-per-gallon20191008.csv). It copies all the files available in the SFTP folder.

Files available in the folder auto-miles-per-gallon20191008.csv auto-miles-per-gallon20191009.csv

Attached the parameters screenshot from Dataset,Pipeline.Dataset parameter[![][1]]2

1

1 Answers

1
votes

I checked File name - @concat('auto-miles-per-gallon',formatDateTime(addDays(utcnow(),-1),'yyyyMMdd'),'.csv'),it is right. I notice that the statements in the SFTP dataset document:

enter image description here

I suggest you configuring the dynamic content for filePath and fileName in the copy activity and try again.

enter image description here