Powershell Get-ChildItem seems to sometimes skip files

Question

I wrote a powershell script to upload files that are created in a local folder to SharePoint with Add-PnPFile.

I loop the script every 20 seconds to check the folder for new files and then upload if the file is not on SharePoint yet. I use a Where-object to select only the last 5 minutes to not have to much files in the loop. (it needs to upload fast since flow is ERP->Local folder-> SharePoint and users are waiting)

This works like 95% of the time but sometimes it skips a files that is existing.

$Localfolder = "D:\DATA"
$A_Files = Get-ChildItem -Path $Localfolder -Include @("*.pdf", "*.csv") | Where-Object {$_.LastWriteTime -gt (Get-Date).AddMinutes(-5)}

If I stop the script and run in the same Powershell session, it doesn't show the file. If I open another powershell session and run the command it shows the file.

Anyone know if there is caching or I'm missing something?

PS C:\Users\rob_admin> $PSVersionTable

Name Value

PSVersion 5.1.14409.1018
PSEdition Desktop
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0...}
BuildVersion 10.0.14409.1018
CLRVersion 4.0.30319.42000
WSManStackVersion 3.0
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1

$Localfolder = "D:\DATA" = $Localfolder = "D:\DATA*" Because I tested if it's not the -Recurse setting — ikhaat
Is it ... a problem? Won't the file get picked up when you run the script again in 20 seconds? — Mathias R. Jessen
Don't poll. Ever. Respond to events instead. Suddenly you have a solution that doesn't burn CPU cycles needlessly. And happens to be reliable. — IInspectable
So the file is missing even if you run Get-ChildItem -Path $Localfolder -Include @("*.pdf", "*.csv") by itself, without the filter? — mklement0
How are these files created? Is there a chance that they are still open in another application, while your script runs? In this case you might even upload incomplete files! — zett42

ikhaat ikhaat · Accepted Answer · 2021-02-04T09:58:34

I start using a different approach which has a 100% success rate after IInspectable comment: $watcher = New-Object System.IO.FileSystemWatcher

So the trigger is now: Register-ObjectEvent $watcher 'Created' -Action $action

Powershell Get-ChildItem seems to sometimes skip files

1 Answers