Filter recent files in Logic Apps' SFTP when files are added/modified trigger

0
votes

I have this Logic App that connects to an SFTP server and it's triggered by the "files are added or modified" trigger. It's set to run every 10 minutes, looking for new/modified files and copying them to an Azure storage account.

The problem is that this SFTP server path is set to overwrite a set of files every X minutes (I have no control over this) and so, pretty often the Logic App overlaps with the update process of these files and downloads files that are still being written. The result is corrupted files.

Is there a way to add a filter to the When files are added or modified (properties only) so that it only takes into consideration files with a modified date of, at least, 1 minute old?

That way, files that are currently being written won't be added to the list of files to download. The next run of the Logic App would then fetch this ignored files and so on.

UPDATE

I've found a Trigger Conditions in the trigger's setting but I can't find any documentation about it. enter image description here

2
azuresftpazure-logic-appslast-modified
If you want to know how to use trigger condition expression, you could refer to this trigger condition, you could filter the property in the triggerbody to decide the logic app run or not. – George Chen
@GeorgeChen Yeah, but that doesn't show all the possible expressions that can be used there. How would you filter by LastModified date there? – empz
I post my answer, you could check it. – George Chen
Any process on this issue? – George Chen

2 Answers

0
votes

According to test the trigger "When files are added or modified", it seems we can not add a filter in the trigger to filter the records which are modified at least 1 minute ago. We can just get the List of Files LastModified datetime and loop them, use "If" condition to judge if we should download it.

enter image description here

Update:

enter image description here

The expression in the screenshot is:

sub(ticks(utcNow()), ticks(triggerBody()?['LastModified']))

Update workaround

Is it possible to add a "Delay" action when the last modified time less than 1 minute ? For example, if the last modified time less than 60 seconds, use "Delay" to wait 5 minutes until the overwrite operation complete, then do the download. enter image description here

0
votes

I check the sample @equals(triggers().code, 'InternalServerError'), actually it uses the condition functions in Logical comparison functions, so the key word is make sure the property you want to filter is in the trigger or triggerBody or you will get the below error.

enter image description here

So I change the expression to like @greater(triggerBody().LastModified,'2020-04-20T11:23:00Z'), this could filter the file modified less than 2020-04-20T11:23:00Z not trigger the flow.

Also you could use other function like less ,greaterOrEquals etc in the Logical comparison functions.