What is the best way to unzip gzip'ed server response in Python 3?

Question

I expected this to work:

>>> import urllib.request as r >>> import zlib >>> r.urlopen( r.Request("http://google.com/search?q=foo", headers={"User-Agent": "Mozilla/5.0 (X11; U; Linux i686) Gecko/20071127 Firefox/2.0.0.11", "Accept-Encoding": "gzip"}) ).read() b'af0\r\n\x1f\x8b\x08...(long binary string)' >>> zlib.decompress(_) Traceback (most recent call last): File "<pyshell#87>", line 1, in <module> zlib.decompress(x) zlib.error: Error -3 while decompressing data: incorrect header check 

But this is not so. Dive Into Python uses StringIO in this example, but it seems to be missing in Python 3. What is the correct way to do this?

Answer 1

It works fine with gzip (gzip and zlib have the same compression, but with different headers / "wrapping. Your error contains this information in the message).

 import gzip import urllib.request request = urllib.request.Request( "http://google.com/search?q=foo", headers={ "Accept-Encoding": "gzip", "User-Agent": "Mozilla/5.0 (X11; U; Linux i686) Gecko/20071127 Firefox/2.0.0.11", }) response = urllib.request.urlopen(request) gzipFile = gzip.GzipFile(fileobj=response) gzipFile.read() 

Answer 2

In Python 3, StringIO is a class in the io module.

So, for the example you contacted if you change:

 import StringIO compressedstream = StringIO.StringIO(compresseddata) 

in

 import io compressedstream = io.StringIO(compresseddata) 

he should work.

Answer 3

For anyone using Python 3.2 or later, there is an even simpler way to decompress a response than any of the answers here:

 import gzip import urllib.request request = urllib.request.Request( "http://example.com/", headers={"Accept-Encoding": "gzip"}) response = urllib.request.urlopen(request) result = gzip.decompress(response.read())