How to create azure data factory pipeline and trigger it automatically whenever file arrive in SFTP?

1
votes

I'm building azure data factory pipeline where source is SFTP and target is azure blob storage. The files can arrive at anytime and any number of files can come into the SFTP on the daily basis. I have to copy the file from Sftp to blob storage whenever any file arrive in SFTP. I know event trigger functionality in ADF but It's possible only if files are coming into the blob storage. Is it possible to achieve same kind of functionality i.e copying files on arrival,when sources are different from blob storage.

1
azureazure-data-factory-2azure-data-factory-pipeline
there's no automatic way for that. You'll need to write something that will monitor your SFTP and trigger ADF pipeline – Thiago Custodio

1 Answers

2
votes

Data Factory can't achieve that.

Some ideas is that you could achieve your purpose with logic app:

  1. You could create a SFTP server trigger: When a file is added or modified
  2. Add an action get a pipeline run to execute the Data Factory pipeline:

enter image description here

Pass the new added filename to the pipeline and run the pipeline.