How to exclude files and folders from Get-ChildItem in PowerShell?

Question

I've made a PowerShell script that runs robocopy with md5 checks.

It works fine, but if I try to exclude some dirs or files, robocopy handles the exclusion, whereas MD5 part of script that compares the hash, doesn't work, returns some errors because the source has more files/hash than destination...

I've tried, maybe, all methods that I've found here and on the Internet! I can't exclude dirs and/or files from a path!

Below is what I've done so far. In this mode, the md5-copy works (without exclusions):

$Source = "F:\"

$IgnoreDir = @(
    $Source + '$RECYCLE.BIN'
    $Source + "System Volume Information"
    $Source + "VMs"
)   
$IgnoreFile = @(
    $Source + "SHDrive.vmdk"
    $Source + "SHDrive-flat.vmdk"
)
$Ignored = $IgnoreDir + $IgnoreFile

Robocopy:

Robocopy.exe /R:1 /W:0 $Source $Dest /E /V /TEE /XD $IgnoreDir /XF $IgnoreFile /LOG:$LogDir\RBCY_MD5_F.txt

MD5:

$SourceHash = Get-ChildItem "$Source\*.*" -Recurse -Force -Exclude $Ignored | Where-Object {!$_.psiscontainer } | Get-FileHash
$SourceHash | Select-Object "Hash", "path" | ft -HideTableHeaders -AutoSize | Out-File -Width "300" $LogDir\SRC_MD5_REF.txt
$SourceHash.Hash | Out-File $LogDir\SRC_MD5.txt

Comparing:

$Diff = Compare-Object -ReferenceObject $(get-content "$LogDir\SRC_MD5.txt") -DifferenceObject $(get-content "$LogDir\DST_MD5.txt")

Content of F:\ drive:

PS C:\Users\Robbi> Get-ChildItem F:\ -force


    Directory: F:\


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d--hs-       19/03/2019     06:40                $RECYCLE.BIN
d-----       16/05/2020     04:41                DATA
d-----       19/01/2020     06:34                Drivers
d-----       16/05/2020     04:55                Gumball
d-----       16/05/2020     04:58                SW
d--hs-       19/03/2019     06:36                System Volume Information
d-----       13/03/2020     16:08                Tools
d-----       12/12/2019     00:02                VMs
d-----       16/05/2020     04:55                _Pre-Cestino
-a----       08/02/2020     03:02    21474836480 SHDrive-flat.vmdk
-a----       08/02/2020     03:02            466 SHDrive.vmdk

How can I exclude the data I don't want to copy from the get-children list? In this specific case and, if possible, in "all cases" in which Get-ChildItem has to exclude an explicit content list (variable string and/or an array) in a whole file system.

mklement0 mklement0 · Accepted Answer · 2020-05-21T12:31:50

As of PowerShell 7.1, the -Exclude and -Include provider parameters of cmdlets such as Get-ChildItem only operate on item names (file / directory names, in the case of the file-system provider), not full paths or directory subtrees.

GitHub feature request #4126 asks for path patterns to be supported too in the future.
GitHub feature request #15159 proposes a new subtree-exclusion parameter, such as
-ExcludeRecursive.

Given that all paths that you want to exclude are direct children of the target directory, I suggest a two-step approach:

# Get all files and directories in $Source, except those to be excluded.
# Note the use of \* instead of \*.*, so as to also include the
# directories (whose names don't have an extension).
$items = Get-Item $Source\* -Force | Where-Object FullName -NotIn $Ignored

# Recursively process all resulting files and directories and
# calculate their hashes.
# Note the use of -File to limit output to files.
$SourceHash = $items | Get-ChildItem -Recurse -Force -File | Get-FileHash

Of course, if you define your $Ignored array in terms of file/directory names only, you could use -Exclude:

# Convert the ignore list to file/directory names only.
$Ignored = $Ignored | Split-Path -Leaf

$SourceHash = Get-ChildItem -File $Source -Recurse -Force -Exclude $Ignored |
                Get-FileHash

If the paths to exclude can occur at any level of the subdirectory hierarchy, more work is needed:

$ignoredRegex = '(?<=^|\{0})({1})(?=\{0}|$)' -f
                  [IO.Path]::DirectorySeparatorChar,
                  ($Ignored.ForEach({ [regex]::Escape($_) }) -join '|')


$SourceHash = Get-ChildItem $Source -Recurse -File -Force |
                Where-Object FullName -notmatch $ignoredRegex
                  Get-FileHash

The above uses a regular expression with the (negated form of the) -match operator to exclude all specified items and their children, recursively, anywhere in the subdirectory tree.

How to exclude files and folders from Get-ChildItem in PowerShell?

1 Answers