ValueError: numpy.dtype is the wrong size, try recompiling

Question

I just installed pandas and the statsmodels package on my python 2.7. When I tried to "import pandas as pd", this error message appears. Can anyone help? Thank!!!

numpy.dtype has the wrong size, try recompiling Traceback (most recent call last): File "<stdin>", line 1, in <module> File "C:\analytics\ext\python27\lib\site-packages\statsmodels-0.5.0-py2.7-win32.egg\statsmodels\formula\__init__.py", line 4, in <module> from formulatools import handle_formula_data File "C:\analytics\ext\python27\lib\site-packages\statsmodels-0.5.0-py2.7-win32.egg\statsmodels\formula\formulatools.p y", line 1, in <module> import statsmodels.tools.data as data_util File "C:\analytics\ext\python27\lib\site-packages\statsmodels-0.5.0-py2.7-win32.egg\statsmodels\tools\__init__.py", li ne 1, in <module> from tools import add_constant, categorical File "C:\analytics\ext\python27\lib\site-packages\statsmodels-0.5.0-py2.7-win32.egg\statsmodels\tools\tools.py", line 14, in <module> from pandas import DataFrame File "C:\analytics\ext\python27\lib\site-packages\pandas\__init__.py", line 6, in <module> from . import hashtable, tslib, lib File "numpy.pxd", line 157, in init pandas.tslib (pandas\tslib.c:49133) ValueError: numpy.dtype has the wrong size, try recompiling 

Answer 1

(to slightly increase my comment)

Numpy developers typically have a backward compatible binary interface (ABI) retention policy. However, ABI does not support transitions.

What does it mean:

A package that uses numpy in a compiled extension compiles against a specific version of numpy. A future version of numpy will be compatible with a compiled package extension (see below for an exception). Distributors of these other packages do not need to recompile their package against newer versions of numpy, and users do not need to update these other packages when users upgrade to a newer version of numpy.

However, this does not go in the other direction. If a package is compiled against a specific version of numpy, say 1.7, then there is no guarantee that the binaries of this package will work with older versions of numpy, say 1.6, and very often or most often they will not.

Binary distribution of packages such as pandas and statsmodels that are compiled against the recent version of numpy will not work if an older version of numpy is installed. Some packages, like matplotlib, if I remember correctly, compile their extensions against the oldest version of numpy that they support. In this case, users with the same old or newer version of numpy can use these binaries.

The error message in the question is a typical result of binary incompatibility.

The solution is to get a binary compatible version, either by updating numpy, or at least the version that pandas or statsmodels were compiled with, or recompiling pandas and statsmodels against an older version of numpy that is already installed.

ABI backward compatibility violation:

Sometimes improvements or refactoring in numpy break ABI backwards compatibility. This happened (unintentionally) with numpy 1.4.0. As a result, users who upgraded numpy to 1.4.0 had binary incompatibilities with all other compiled packages that were compiled against a previous version of numpy. To do this, all packages with binary extensions using numpy must be recompiled to work with an incompatible version of ABI.

Answer 2

For me (Mac OS X Maverics, Python 2.7)

 easy_install --upgrade numpy 

helped. After that, you can install the updated pandas, scikit-learn, etc packages using pip :

 pip install pandas 

Answer 3

I found this to be a simple version, outdated or inappropriate, and was fixed with:

 pip install --upgrade numpy pip install --upgrade scipy pip install --upgrade pandas 

Or it can work with one liner:

 pip install --upgrade numpy scipy pandas 

Answer 4

I had a similar error with another library, and I realized that several versions of numpy were installed on my system. The fix for me was to edit my PYTHONPATH and put the site packages containing the latest version of numpy in the first position.

Answer 5

As in here , for me only sudo pip install pandas==0.13.1 worked

Answer 6

I also encounter this error when using pandas to access MYSQL. This error message indicates a binary compatible issue and can be resolved using the latest pandas and numpy packages. Here are my steps to solve this problem, and it works well on my Ubuntu 12.04:

 cd /tmp/ wget https://pypi.python.org/packages/source/p/pandas/pandas-0.12.0.tar.gz tar xzvf pandas-0.12.0.tar.gz cd pandas-0.12.0 easy_install --upgrade numpy 

Answer 7

In my case, I installed pandas -0.10.0.win-amd64-py2.7, but checked if the error was fixed in a later version of pandas. So I did easy_install -U to force the upgrade, but then got the above error due to some incompatibilities with numpy, etc .... when I did

 import pandas 

To fix, I just reinstalled the pandas -0.10.0.win-amd64-py2.7 binary and everything works. I have not seen this answer (suggests using pip), which may have helped me (though not sure) Install a specific version using easy_install

It also emphasizes why you need to use virtualenv (which I was not).

Answer 8

For me (Mac OS X Mavericks) it worked to install the version for python2.6:

 sudo port install py26-scikit-learn 

then do:

 python2.6 myscript.py 

Answer 9

The problem I solved in Webfaction is the old numpy library (1.5), which contradicted my new

pip install pandas

installation in .virtualenv.

The problem was solved after I made pip install pandas from a virtual environment. The idea came from a discussion https://github.com/pydata/pandas/issues/3711 , thanks, cpcloud !

Answer 10

I just meet this "ValueError" problem and addressed it. There is definitely something wrong with the numpy package.

But when I try to run pip install --upgrade numpy , it failed, so I delete and upload the new numpy.zip file. Then manually unzip and python setup.py install it.

Fortunately, it works!

Answer 11

Like @ user333700, the required library versions may not match each other. You get one library as another dependency. Then, not knowing that it is already installed as a dependency, you need this particular library, and you install one version. With these methods, addictions can go bad.

I lived this way and was looking for a solution. Found: stack overflow

I had two different versions for the egg info file and the numpy folder name:

 drwxr-xr-x. 19 root root 4096 Sep 25 15:00 numpy drwxr-xr-x. 2 root root 4096 Sep 22 11:25 numpy-1.13.1.dist-info -rw-r--r--. 1 root root 1630 Nov 20 2015 numpy-1.7.1-py2.7.egg-info 

I uninstalled them all and reinstalled numpy using pip.