Download Python 2..py file in Python 3

Question

I am trying to download /usr/share/matplotlib/sample_data/goog.npy :

 datafile = matplotlib.cbook.get_sample_data('goog.npy', asfileobj=False) np.load(datafile) 

This is fine in Python 2.7, but throws an exception in Python 3.4:

 UnicodeDecodeError: 'ascii' codec can't decode byte 0xd4 in position 1: ordinal not in range(128) 

I assume this has something to do with the bytes/str/unicode mismatch between Python 2 and 3, but has no idea how to get through.

Question:

• How to download a .npy file (NumPy data) from Python 2 to Python 3?

Answer 1

The problem is that the file contains serialized (pickled) Python date and time objects, not just numeric data. The Python serialization format for these objects is incompatible between Py2 and Py3:

 python2 >>> import pickle >>> pickle.dumps(datetime.datetime.now()) "cdatetime\ndatetime\np0\n(S'\\x07\\xde\\x06\\t\\x0c\\r\\x19\\x0f\\x1fP'\np1\ntp2\nRp3\n." 

and

 python3 >>> import pickle >>> pickle.loads(b"cdatetime\ndatetime\np0\n(S'\\x07\\xde\\x06\\t\\x0c\\r\\x19\\x0f\x1fP'\np1\ntp2\nRp3\n.") Traceback (most recent call last): File "<stdin>", line 1, in <module> UnicodeDecodeError: 'ascii' codec can't decode byte 0xde in position 1: ordinal not in range(128) 

The workaround is to change inside Numpy code

 numpy/lib/format.py: ... 446 array = pickle.load(fp) 

before array = pickle.load(fp, encoding="bytes") . The best solution would be to allow numpy.load on the encoding parameter.

Answer 2

In python 3.5 with numpy 1.10.4 the following command works for me:

 D = np.load(file, encoding = 'latin1') 

It does not work with the same error message when I do not specify the encoding.

Answer 3

One way that helped me was to dump the numpy array loaded in python2. *, to the csv file, and then read it back into python3. *

 # Dump in python2 import numpy as np datafile = matplotlib.cbook.get_sample_data('goog.npy', asfileobj=False) arr = np.load(datafile) np.savetxt("np_arr.csv", arr, delimiter=",") 

Now read the file back in python3

 # Load in python3 import numpy as np arr = np.loadtxt(open("np_arr.csv"), delimiter=",")