Geek Answers Handbook

Copy millions of files

I have about 3 million files that I need to copy from one folder to another on top of my SAN. What is the best way for me to do this?

+3
source share
6 answers

If you are in windows, use robocopy. It is very reliable and suitable for such situations. It supports dead link detection and may ask for duplicate copies if they are interrupted.

+6
source

If the direct copy is too slow (although the SAN with writeback cache will be about as fast as anything for this type of operation), you can deactivate the files in one or more archives, and then expand the archives to the destination. This will slightly reduce disk tear.

tar cpio, stdout, tar/cpio, .

tar :

tar cf - * | (cd [destination dir] ; tar xf - )

SAN .

+7

rsync? , .

+4

Microsoft SyncToy . .

+2

Teracopy , .

http://www.codesector.com/teracopy.php

, * nix, cuteftp.

+1

If you ask me, this is simply the best way to copy using clean software.

Just something like:

cp -pvr /pathtoolddir /pathtonewdir

in linux box will work and work fine. Any compression between them will simply slow down the process.

0
source

More articles:


All Articles