Compressing a folder with many duplicate files

Question

I have a rather large folder (~ 10 GB) that contains a lot of duplicate files all over the tree directory. Many of these files are duplicated 10 times. Duplicate files are not next to each other, but in different subdirectories.

How can I compress a folder to make it small enough?

I tried using Winrar in "Best" mode, but it did not compress it at all. (Pretty weird)

Will zip \ tar \ cab \ 7z \ any other compression tool work better?

I do not mind allowing the tool to work for several hours - but no more.

I prefer not to do this programmatically myself

Answer 1

WinRAR . , .

. WinRAR Solid archives. , .

. , 327 22 453 , 47 . 327 , , RAR 4 , 193 . , , .

rarfiles.lst . , , . WinRAR .

" " GUI- WinRAR. / . , - , *.png, *.jpg, *.zip, *.rar... , . *.jpg , .

: , Total Commander, UltraFinder UltraCompare , , , , , .

Answer 2

7-zip WIM, "" . 7-zip GUI, "wim".

7-zip, . . https://serverfault.com/questions/483586/backup-files-with-many-duplicated-files

Answer 3

7-zip. :

7za a -r -t7z -m0=lzma2 -mx=9 -mfb=273 -md=28 -ms=8g -mmt=off -mmtf=off -mqs=on -bt -bb3 archife_file_name.7z /path/to/files

a -

-r - Recurse

-t7z - (7z )

-m0=lzma2 - LZMA2. LZMA 7z. LZMA:

• ( 4 )
• : 1 / 2
• : 10-20 / 2
• ( )
• : 5
• P4

-mx=9 - . x = 0 ( ). = 9 -

-mfb=273 - LZMA. 5 273. 32 64 . .

-md=29 - LZMA. , . 1536 , 32- 7-Zip 128 . LZMA: 24 (16 ) , 25 (32 ) (-mx = 7) 26 (64 ) - (-mx = 9). - [b | k | m | g], DictionarySize = 2 ^ . , LZMA N, N (RAM).

md=29 16 . 7-zip 5 . - .

-ms=8g - . s=on. . . , .

. .7z , .

-mmt=off - OFF. , 7-zip- . . , .

-mmtf=off - OFF.

-myx=9 - , ( ).

-mqs=on - . .

-bt - -bb3 -

Answer 4

3 , ( Windows):

1. 7zip LZMA2 1536
2. WinRar ""
3. WIM 7zip

10 - ( , .php,.html,.js,.css,.jpeg,.sql ..) 1 ( 100 ). 7zip WinRar 400/500 , (1) 80 , (2) 100 (3) 170 .

: @Griffin 7zip LZMA2 ( , , ) WIM 7zip. , , , WIM 70% . 2 , , 1.

Answer 5

? 64 ? ( ), tar, .tar, xz , .tar.xz. , .tar xz, . "DictSize" xz man. 256 64 .

Answer 6

eXdupe www.exdupe.com, , /