Geek Answers Handbook

PowerShell foreach file in a folder using a lot of memory

I am new to PowerShell and programming, so I am sure that this code can be further optimized.

However, my biggest problem with this code is that it puts data from the foreach loop into memory. What is the best way to prevent this?

I started rewriting this script from a script that I found here: PowerShell: delete files older than x days

Edit: Oh, there are about 60,000 ~ files in each TargetFolder.

## Set Window Title $title = $Host.UI.RawUI.WindowTitle $titdef = $title $Host.UI.RawUI.WindowTitle = "Checking files. Do NOT close " + "($title)" ## Debug (0=Nothing, 1=Extended log, 2=Extended log+Screen) $Debug = "2" ## Log file $log = "C:\status\status.txt" Add-Content $log "nul" ## Clear log before adding new data Remove-Item $log ## Folders to check $TargetFolder1 = "\\server\c$\BiztalkProjects\UDC\output\ume" $TargetFolder2 = "\\server\c$\BiztalkProjects\UDC\output\upp" ## Required stuff $Now = Get-Date $Days = "2" $TwoDays = $Now.AddDays(-$Days) $Folder1 = get-childitem $TargetFolder1 -include * $Folder2 = get-childitem $TargetFolder2 -include * ## Window Message "Checking for files newer than $Days days in:" "$TargetFolder1" "$TargetFolder2" "" "This may take 5-15 minutes . . . (Will commit about 250MB~ RAM during the period)" ## Reset variable $OK to "0" $OK = 0 if ($Debug -eq "2") {"Check1"} foreach ($File in $Folder1) { $FileLastWrite = $File.LastWriteTime if ($File.LastWriteTime -le $TwoDays) { $OK = $OK + "0" if ($Debug -eq "2") { "OK + 0 (=$OK) $File ($FileLastWrite)" } } else { $OK = $OK + "1" if ($Debug -eq "2") { "OK + 1 (=$OK) $File ($FileLastWrite)" } } } if ($Debug -eq "2") {"Check2"} foreach ($File in $Folder2) { $FileLastWrite = $File.LastWriteTime if ($File.LastWriteTime -le $TwoDays) { $OK = $OK + "0" if ($Debug -eq "2") { "OK + 0 (=$OK) $File ($FileLastWrite)" } } else { $OK = $OK + "1" if ($Debug -eq "2") { "OK + 1 (=$OK) $File ($FileLastWrite)" } } } if ($OK -gt "0") { if ($Debug -eq "2") { "" "Found $OK file(s) newer than $Days days" } Add-Content $log "OK" if ($Debug -ge "1") { Add-Content $log "Found $OK file(s) newer than $Days days" } } else { if ($Debug -eq "2") { "" "Found $OK file(s) newer than $Days days" } Add-Content $log "Error" if ($Debug -ge "1") { Add-Content $log "Found $OK file(s) newer than $Days days" } } $Host.UI.RawUI.WindowTitle = $titdef

This version works. Thanks to everyone who helped!

 ## Set Window Title $title = $Host.UI.RawUI.WindowTitle $titdef = $title $Host.UI.RawUI.WindowTitle = "Checking files. Do NOT close " + "($title)" ## Debug (0=Nothing, 1=Extended log, 2=Extended log+Screen) $Debug = "2" ## Log file $log = "C:\status\status.txt" Add-Content $log "nul" ## Clear log before adding new data Remove-Item $log ## Folders to check $TargetFolder1 = "\\server\c$\BiztalkProjects\UDC\output\ume" $TargetFolder2 = "\\server\c$\BiztalkProjects\UDC\output\upp" ## Required stuff $Now = Get-Date $Days = "2" $TwoDays = $Now.AddDays(-$Days) ## Window Message "Checking for files newer than $Days days in:" "$TargetFolder1" "$TargetFolder2" "" "This may take 5-15 minutes . . ." ## Reset variable $OK to "0" $OK = 0 get-childitem $TargetFolder1,$TargetFolder2 -filter *.xml |where-object { $FileLastWrite = $_.LastWriteTime if ($FileLastWrite -le $TwoDays) { $OK = $OK + "0" } else { $OK = $OK + "1" } } if ($OK -gt "0") { if ($Debug -eq "2") { "" "Found $OK file(s) newer than $Days days" } Add-Content $log "OK" if ($Debug -ge "1") { Add-Content $log "Found $OK file(s) newer than $Days days" } } else { if ($Debug -eq "2") { "" "Found $OK file(s) newer than $Days days" } Add-Content $log "Error" if ($Debug -ge "1") { Add-Content $log "Found $OK file(s) newer than $Days days" } } $Host.UI.RawUI.WindowTitle = $titdef
+4
source share
2 answers

Try:

  $ Folder1 = get-childitem $ TargetFolder1 -filter * .xml

Much more effective than using include (in a large set of files). Or even better:

  get-childitem $ TargetFolder1 -filter * .xml |  % {}

EDIT

Now I see your problems are related to include ! If you need to use wildcards and regular expressions to get a child, use filter instead. It is much more effective because it does not use a regular expression.

Try:

  $ Folder1 = get-childitem $ TargetFolder1 -filter *

even if it’s not entirely clear what you are filtering there (or want to include) ... is that all? Maybe you don't need a template at all.

In my humble opinion, the best way to prevent a foreach loop in memory is by piping. However, due to the nature of your script (many nested controls inside the loop), this is not so simple.

Try:

  $ Folder1 |  % {
   # ...
 }

even if I doubt it will improve performance.

or

  get-childitem $ TargetFolder1 -include * |  % {
   # ...
 }
+4
source

Sort of

  $TS=(Get-Date).AddDays(-2) $COUNT = (get-childitem -recurse path1,path2 |where-object {$_.LastWriteTime -gt $TS}|measure-object).Count if ($COUNT > 0) { Echo "There are $COUNT files" }

could be better (I have not tested it for performance)

+4
source

More articles:


All Articles