Are zlib.compress in Python and Deflater.deflate in Java (Android) compatible?

Question

I am porting a Python application to Android, and at some point this application should contact the web service, sending compressed data.

To do this, use the following method:

def stuff(self, data): "Convert into UTF-8 and compress." return zlib.compress(simplejson.dumps(data)) 

I use the following method to try to imitate this behavior in Android:

 private String compressString(String stringToCompress) { Log.i(TAG, "Compressing String " + stringToCompress); byte[] input = stringToCompress.getBytes(); // Create the compressor with highest level of compression Deflater compressor = new Deflater(); //compressor.setLevel(Deflater.BEST_COMPRESSION); // Give the compressor the data to compress compressor.setInput(input); compressor.finish(); // Create an expandable byte array to hold the compressed data. // You cannot use an array that the same size as the orginal because // there is no guarantee that the compressed data will be smaller than // the uncompressed data. ByteArrayOutputStream bos = new ByteArrayOutputStream(input.length); // Compress the data byte[] buf = new byte[1024]; while (!compressor.finished()) { int count = compressor.deflate(buf); bos.write(buf, 0, count); } try { bos.close(); } catch (IOException e) { } // Get the compressed data byte[] compressedData = bos.toByteArray(); Log.i(TAG, "Finished to compress string " + stringToCompress); return new String(compressedData); } 

But the HTTP response from the server is incorrect, and I think this is because the compression result in Java does not match the result in Python.

I did a little test compressing "a" with both zlib.compress and deflate.

Python, zlib.compress () → x% 9CSJT% 02% 00% 01M% 00% A6

Android, Deflater.deflate → H% EF% BF% BDK% 04% 00% 00b% 00b

How do I compress data on Android to get the same zlib.compress () value in Python?

Any help, guide or guide is appreciated!

Answer 1

Although they are not exactly the same algorithms, they seem to be fully compatible (this means that if you compress, for example, a line using Deflater.deflate, you can correctly unzip it using zlib).

What caused my problem was that all form variables in POST should be skipped, and the Android application does not. Encoding data on Base64 before sending it and changing the server to decode it using Base64 before unpacking using zlib, solving the problem.

Answer 2

compression and deflation are different compression algorithms, so the answer will be incompatible. As an example of the difference here, “a” is compressed using two algorithms via Tcl:

 % binary encode hex [zlib compress a] 789c4b040000620062 % binary encode hex [zlib deflate a] 4b0400 

Your python code really does the compression. And the android code does deflate, however you also get the UTF-8 byte order mark added to the Android version (\ xef \ xbf \ xbf)

You can emit data deflation using python:

 def deflate(data): zobj = zlib.compressobj(6,zlib.DEFLATED,-zlib.MAX_WBITS,zlib.DEF_MEM_LEVEL,0) zdata = zobj.compress(data) zdata += zobj.flush() return zdata 

 >>> deflate("a") 'K\x04\x00' 

Answer 3

Does byte[] input = stringToCompress.getBytes("utf-8"); ? If your default encoding for the platform is not UTF-8, this will force String → bytes to use UTF-8. In addition, the same goes for the last line of your code, where you create a new String - you can explicitly specify UTF-8 as the encoding encoding.