Unrar archive on boot

Question

I have a program that loads part01, then part02, etc. rar file broken over the Internet. My program first loads part 01, then part02, and so on. After some tests, I found out that using, for example, UnRAR2 for python, I can extract the first part of the file (.avi file) contained in the archive, and I can play it in the first minutes. When I add another file, it extracts a little more and so on. I wonder: is it possible to extract individual files when they are downloaded? I need to start extracting part01 without waiting for the download to complete ... maybe?

Thank you very much!

Matteo

Answer 1

You are talking about the .avi file inside rar archives. Are you sure archives are actually compressed? Video files released by the warez scene do not use compression:

Ripped movies are still packed due to the large file size, but compression is prohibited, and the RAR format is used only as a container. Because of this, modern playback software can easily play a release directly from packaged files and even transfer it as the version loads (if the network is fast enough).

(I think VLC, BSPlayer, KMPlayer, Dziobas Rar Player, rarfilesource, rarfs, ...) You can check the compression as follows:

• Open the first .rar archive in WinRAR. (name.part01.rar or name.rar for old-style volume names)
• Click the info button.

If the version to extract indicates 2.0, then the archive does not use compression. (if you do not have ten-year ranks). You can see that the total size and package size will be equal.

Is it possible to make it extract individual WHILE files loading them?

Yes. If compression is not used, you can write your own program to extract files. (I know someone who wrote a script to directly download a movie from external rar files, but it's not publicly available and I don't have one.) Since you mentioned Python, I suggest you take a look at Marko Kreen's rarfile 2.2 , how by pyarrfs . An archive is just a file cut into headers (rar blocks). This will be a copy operation that must be paused until the next archive is downloaded.

I strongly believe that this is also possible for compressed files. Your approach here will be different because you must use unrar to extract the compressed files. I must add that there is also a free RARv3 implementation for rars extraction implemented in Unarchiver.

I think this option for (un) rar will allow:

-vp Pause before each volume By default RAR asks for confirmation before creating or unpacking next volume only for removable disks. This switch forces RAR to ask such confirmation always. It can be useful if disk space is limited and you wish to copy each volume to another media immediately after creation. 

This will give you the option to pause the extraction until the next archive is downloaded.

I believe that this will not work if rar was created with the "solid" option enabled.

If a solid option is used for rars, all packed files are treated as one large file stream . This should not cause any problems if you always start with the first file, even if it does not contain the file you want to extract. I also think that it will work with password-protected archives.

Answer 2

I really doubt. By the nature of compression (from my understanding), every bit is needed to unclench it. It seems that the source of where you are loading has deliberately broken avi into pieces before compression, but by the time you apply compression, everything you have compressed is now one atomic unit. Therefore, they kindly broke the whole avi into parts, but each part is still an atomic thread.

But I am not an expert in compression.

The only test I can think of right now is something like: curl http://example.com/Part01 | unrar curl http://example.com/Part01 | unrar .

Answer 3

I don’t know if this was set based on a specific language, but you can translate a compressed RAR directly from the Internet and unpack it on the fly. I can do this with my C # library http://sharpcompress.codeplex.com/

The RAR format actually looks nice. It has headers preceding each record, and the compressed data itself does not require random access to the byte stream.

Make it multicomponent files, you will need to fully extract part 1 first, and then continue to write when part 2 is available.

All this is possible using my RarReader API. A solid archive is also thread safe (in fact, it is only thread safe. It is not possible to randomly access files in a reliable archive. You pretty much have to extract them all at once.)