python - ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject

Question

Importing from pyxdameraulevenshtein gives the following error, I have

pyxdameraulevenshtein==1.5.3, 
pandas==1.1.4 and 
scikit-learn==0.20.2. 
Numpy is 1.16.1. 
Works well in Python3.6, Issue in Python3.7.

Has anyone been facing similar issues with Python3.7 (3.7.9), docker image - python:3.7-buster

__init__.pxd:242: in init pyxdameraulevenshtein
    ???
E   ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject

Answer 1

I'm in Python 3.8.5. It sounds too simple to be real, but I had this same issue and all I did was reinstall numpy. Gone.

pip uninstall numpy
pip install numpy

Answer 2

try with numpy==1.20.0 this worked here, even though other circumstances are different (python3.8 on alpine 3.12).

Answer 3

Indeed, (building and) installing with numpy>=1.20.0 should work, as pointed out e.g. by this answer below. However, I thought some background might be interesting -- and provide also alternative solutions.

There was a change in the C API in numpy 1.20.0. In some cases, pip seems to download the latest version of numpy for the build stage, but then the program is run with the installed version of numpy. If the build version used in <1.20, but the installed version is =>1.20, this will lead to an error.

(The other way around it should not matter, because of backwards compatibility. But if one uses an installed version numpy<1.20, they did not anticipate the upcoming change.)

This leads to several possible ways to solve the problem:

• upgrade to numpy>=1.20.0
• use minmum supported numpy version in pyproject.toml (oldest-supported-numpy)
• install with --no-binary
• install with --no-build-isolation

For a more detailed discussion of potential solutions, see https://github.com/scikit-learn-contrib/hdbscan/issues/457#issuecomment-773671043.

Answer 4

I had this issue when using the tensorflow object api. Tensorflow is currently NOT compatible with numpy==1.20 (although this issue is not apparent until later). In my case, the issue was caused by pycocotools. I fixed by installing an older version.

pip install pycocotools==2.0.0

Answer 5

For anyone using Poetry it is necessary to have experimental.new-installer set to true for an application with a numpy<1.20 dependency to be built correctly i.e:

poetry config experimental.new-installer true

It is true by default but if (as was the case for me) it has been changed it can catch you out.

My application uses Tensorflow and I did not therefore have the option of upgrading to >1.20. Poetry also does not support --no-binary dependencies.

Answer 6

After you pip install any package, makes sure you restart the Kernel and should work. usually packages get upgraded automatically and all you need is a quick restart. At least, this what worked in my situation and I was getting the same error when I tried to install and use pomegranate.

Answer 7

For almost the same image : python:3.7-slim-buster

I started to have this problem just today, it was non exitent before.

I solved it by removing numpy from requirement.txt file and doing instead the following in my Dockerfile:

RUN pip3 install --upgrade  --no-binary numpy==1.18.1 numpy==1.18.1 \
&& pip3 install -r requirements.txt 

I use some old versions of keras and its librairies and updrading to numpy 1.20.0 didn't work for those librairies. But I think the solution consist in the first command that I gave you wich tell pip to try to not compile numpy and download a pre-compiled version.

The trick in the command is that you might find people telling you to use --no-binary option of pip to solve the problem, but they don't specify how and it might be tricky (as it happened to me); you have to write the package two times in the command in order for it to work or else pip will throw you an error.

I think the --upgrade option in the first command isn't necessary.