I have CMD code that Rojo and Magoo helped me write that works with some XML files in a directory. The code captures the date and time from the files in the file name and creates the year and month from it, and then moves the files to them. The problem that I encountered is that the folder itself contains 914,000 xml files, and the script simply cannot process it. I need something faster or a multi-threading script method. Another option that I considered is to move several thousand files at a time and just run it on those from the temp directory and at the very end of the script move these folders to the production location. Here is the code and another script for creating XML files for testing. The date is not checked, but they should not be for this exercise. This will work on a Microsoft Server 2012 R2 virtual machine.(R) Xeon (R) CPU E5-2650 0 @ 2.00GHz, 2000 Mhz, 1 Core (s), 1 logical processor and 4 gigabytes. I also include Powershell and VbScript tags if anyone can offer any recommendations for writing code in these languages.
moving xml script
@ECHO OFF
SETLOCAL
Title Reorganizing XMLs - DO NOT CLOSE THIS WINDOW!
color 0F
mode con: cols=100 lines=6
prompt $t $d$_$p$g
::Get start time
for /F "tokens=1-4 delims=:.," %%a in ("%time%") do (
set /A "start=(((%%a*60)+1%%b %% 100)*60+1%%c %% 100)*100+1%%d %% 100"
)
Echo Start time: %start%
set "sourcedir=C:\Temp\TestDummyFiles"
set "tempdir=C:\temp\xmlreorgtemp"
::call :Get1000Files %sourcedir% %tempdir% %total%
pushd %sourcedir%
SET "spinChars=\|/-"
for /f %%a in ('"prompt $H&for %%b in (1) do rem"') do set "BS=%%a"
set "spaces= "
SET /a filesMoved = 0, spinPos = 0, prev = 0
echo Moving XML Files...
setlocal enabledelayedexpansion
for /L %%I in (1,1,7) do set "BS=!BS!!BS!"
for /L %%I in (1,1,3) do set "spaces=!spaces!!spaces!"
For %%A in (%sourcedir%\*.xml) do set /a cnt+=1
echo.
Echo Total XML files: %cnt%
echo.
FOR /f "tokens=1*delims=" %%a IN ('dir /b /a-d "%sourcedir%\*.xml" ' ) DO (
set /a filesmoved += 1
call :spinner !filesmoved! "%%~nxa"
)
call :spinner %filesMoved% Done.
for /F "tokens=1-4 delims=:.," %%a in ("%time%") do (
set /A "end=(((%%a*60)+1%%b %% 100)*60+1%%c %% 100)*100+1%%d %% 100"
)
echo End time: %end%
set /A elapsed=end-start
rem Show elapsed time:
set /A hh=elapsed/(60*60*100), rest=elapsed%%(60*60*100), mm=rest/(60*100), rest%%=60*100, ss=rest/100, cc=rest%%100
if %mm% lss 10 set mm=0%mm%
if %ss% lss 10 set ss=0%ss%
if %cc% lss 10 set cc=0%cc%
echo Elapsed Time: %hh%:%mm%:%ss%
endlocal & echo;
exit /b 0
:Get1000Files
@echo off
setlocal enabledelayedexpansion
for /f %%a in ('dir "%~1" /b /a-d *.xml') do (
set /a cnt+=1 & move "%%~a" "%~2"
if !cnt! EQU 1000 exit /b
)
exit /b
:spinner <filecount> <filename>
set /a spinPos += 1, spinPos %%= 4, ten = %~1 / 10 * 10
if "%~2"=="Done." set ten=%~1
set "str=[!spinChars:~%spinPos%,1!] %ten% files moved... [%~2]"
set "str=%str:~0,79%"
call :length len "%str%"
set /a diff = 79 - len
if %diff% gtr 0 set "str=%str%!spaces:~-%diff%!"
set /P "=!BS:~-79!%str%"<NUL
if "%~2" NEQ "Done." call :process %~2
exit /b 0
:length <return_var> <string>
setlocal enabledelayedexpansion
if "%~2"=="" (set ret=0) else set ret=1
set "tmpstr=%~2"
for %%I in (4096 2048 1024 512 256 128 64 32 16 8 4 2 1) do (
if not "!tmpstr:~%%I,1!"=="" (
set /a ret += %%I
set "tmpstr=!tmpstr:~%%I!"
)
)
endlocal & set "%~1=%ret%"
exit /b 0
:process
FOR /f "tokens=2,3,6delims=_" %%m IN ("%~1") DO SET "date1=%%m"&SET "date2=%%n"&SET "whichdate=%%o"
IF DEFINED whichdate SET "date1=%date2%"
IF NOT DEFINED date2 exit /b 1
If not exist .\%date1:~0,4%\%date1:~4,2% MD .\%date1:~0,4%\%date1:~4,2%
MOVE %~1 .\%date1:~0,4%\%date1:~4,2%\ > nul
And a script to create some dummy files
@echo off
setlocal EnableDelayedExpansion
cd /d %~dp0
For /f %%a in ('copy /Z "%~dpf0" nul') Do set "CR=%%a"
set fileSize=%~Z1
set /a cnt=0
echo Creating files. Please wait.&echo.
:loop
if %cnt% GTR 5000 exit /b
set /a cnt+=1
set /p "=Creating %cnt% File(s) !CR!"<nul:
Call :random 2009 2015 yyyy
call :random 1 12 mm
call :random 1 31 dd
if %mm% LSS 10 set mm=0%mm%
if %dd% LSS 10 set dd=0%dd%
set /P "=0" > thisSize.txt < NUL
(for /L %%i in (0,1,30) do (
set /A "bit=(1<<%%i)&fileSize, fileSize&=~(1<<%%i)"
if !bit! neq 0 type thisSize.txt
if !fileSize! neq 0 type thisSize.txt >> thisSize.txt
)) > IDABCDEFG001_STUFF_%yyyy%%mm%%dd%_ABC_0_1234567890.xml
del thisSize.txt
goto :loop
exit /b
:random Min Max [RtnVar]
@echo off & setlocal
set /a rtn=%random% %% ((%~2)-(%~1)+1) + (%~1)
(endlocal
if "%~3" neq "" (set %~3=%rtn%) else echo:%rtn%
)
exit /b
The server has Powershell 4.