Change python byte type to string

Question

I am using python to work with the stackoverflow API. I run the following commands:

f = urllib.request.urlopen('http://api.stackoverflow.com/1.0/stats') d = f.read() 

Type d is the bytes class, and if I print it, it looks like this:

 b'\x1f\x8b\x08\x00\x00\x00 .... etc 

I tried d=f.read().decode('utf-8') as it is the encoding indicated in the header, but I get the 'utf8' codec can't decode byte 0x8b in position 1" error message

How to convert byte object that I got from my urllib.request call to string?

Answer 1

Make sure the response body is not loaded. Believe your transfer encoding or such for the response header, I am sure that you are dealing with compressed data, not character encoding.

update: Realizing that I have a bad habit of not explaining / providing enough details. For Python gzip'd byte strings, they always start with 1f8b. Someone explains it better here fooobar.com/questions/255486 / ...