How to simulate the use_crc (hacked) 1998 uncompress () flag in the 2013 zlib API?

Question

I updated the project code from the 1998 zlib version to the 2013 zlib version. One thing that seems to have changed is that the use_crc flag was used in the uncompress function, which seemed to be missing:

int ZEXPORT uncompress (dest, destLen, source, sourceLen, use_crc) Bytef *dest; uLongf *destLen; const Bytef *source; uLong sourceLen; int use_crc; // <-- vanished (?) 

( UPDATE: as @Joe pointed out, this is probably a third-party modification . Accordingly updated. The rest of the question is still applicable, as in: “How do I best do this with today's zlib stock.”)

In the code I'm learning, uncompress () is called by deconstructing the binary .zip format and passing the "payload" of the data. The code passed the crc flag as 1. If the flag was not used, it will receive Z_DATA_ERROR (-3). (A zlib without the use_crc flag gets Z_DATA_ERROR just as if the flag were false.)

In experiments, I found that very small files worked without use_crc. Then the small counting files switched to non-working between "12345678901234" and "123456789012345" . The reason was that the first file that was photographed instead of the saved uncompressed (which zip code is called “6%” savings)

In trying to get zlib to accept it, I tried a lot of things. This included an attempt to 16 + MAX_WBITS . Nothing seemed to handle the payload from zip test.zip test.txt , as the old code did.

If I were ready to subtract one of my destination size, I seemed to be able to suppress the erroneous check ... when losing one byte. Here's a simple test program with minimal zip hardcoded payload:

 #include <stdio.h> #include "zlib.h" int main(int argc, char *argv[]) { char compressed[] = { 0x78, 0x9C, 0x33, 0x34, 0x32, 0x36, 0x31, 0x35, 0x33, 0xB7, 0xB0, 0x34, 0x30, 0x04, 0xB1, 0xB8, 0x00, 0x31, 0x30, 0xB1, 0x30, 0x10, 0x00, 0x00, 0x00 }; // last 4 bytes are size (16) char uncompressed[16 + 1]; // account for null terminator int ret; z_stream strm; memset(uncompressed, 'X', 16); uncompressed[16] = '\0'; strm.zalloc = strm.zfree = strm.opaque = Z_NULL; strm.total_out = 0; strm.avail_in = 25; strm.next_in = compressed; ret = inflateInit2(&strm, MAX_WBITS /* + 16 */); // it is Z_OK strm.avail_out = 15; // 16 gives error -3: "incorrect header check" strm.next_out = uncompressed; ret = inflate(&strm, Z_NO_FLUSH); if (ret != /* Z_STREAM_END */ Z_OK) { // doesn't finish... printf("inflate() error %d: %s\n", ret, strm.msg); return 2; } inflateEnd(&strm); printf("successful inflation: %s\n", uncompressed); return 0; } 

Output:

successful inflation: 123456789012345X

Data mapping occurs without compression, but we need all 16 bytes. (There is a new line from the file to be received.) 16 + MAX_WBITS cannot even get this.

Any ideas what goes wrong? No permutation of the settings seems to go to errors.

Answer 1

No, there were no incompatible changes to the zlib interface since it was introduced more than 20 years ago. For uncompress() there was never a use_crc argument.

The example you give is a two-byte zlib header, data with deflation of compressed data, CRC-32 of deflation data in big-endian order followed by a four-byte length in little-endian order. This is a truly weird blur of zlib and gzip covers and has nothing to do with the zip format that you continue to mention. (What do you mean by "payloads inside zip files"?). Zlib has Adler-32 at the end in ordinal order, while gzip has CRC-32 in order, followed by a four-byte length in a small, endian order. This mixes them up, including the byte order, and then deliberately misleads the actual zlib header on the subject, which is an insult to everything that is good and decent in this world.

I am sure that the one who invented this format was drunk at that time.

To decode this, you need to:

• Discard the first two bytes of the stream. (You can verify that this is a valid zlib header, but that doesn't make sense when interpreting the rest of the stream.)

• Use raw deflate, initializing with inflateInit2(&strm, -15) , to decompress the data. When unpacking, keep track of the total length and calculate CRC-32 with crc32() .

• After the deflation data is complete, read the next four bytes, collect them in big-endian order to a 32-bit value, and compare them with the calculated CRC-32. If it does not match, the stream is corrupt or not one of these oddly formatted streams. (Perhaps try again, decrypt it as a regular zlib stream. If it has a good zlib header, then maybe this is what it really is, unlike one of these Frankenstein streams.)

• Read the next four bytes and collect them in little-endian order and compare them with the length of the uncompressed data. If it does not match, then the thread is damaged, or it is not what you think.

• If the data doesn't end here, something else odd happens. See a drunk person.