2
votes

Background:

Using mac 10.15, I just installed python, conda and julia.

I then used conda to install numpy and installed julia PyCall pointing ENV["PYTHON"] to the conda executable (out put of sys.executable in python and the desired conda enviorment).

I have a python library "mylib" that uses numpy

Problem

When I tried to

pyimport("mylib")

I got

Intel MKL FATAL ERROR: Cannot load libmkl_intel_thread.dylib.

which I discovered is due to a conflict between 64bit MKL library julia uses and the 32bit MKL library python uses:

https://www.reddit.com/r/Julia/comments/jj7ubh/pycall_intel_mkl_error/

https://github.com/JuliaPy/PyCall.jl/issues/443

The solutions given in the julia forms suggest recompiling julia with a changed flag. This seems unnecessary painful, is there another option?

1
my guess is that you were trying to use 32-bit anaconda. Normally MKL comes in 64-bit version anaconda.org/anaconda/mkl - Przemyslaw Szufel

1 Answers

1
votes

The best solution I found, was to create a seperate conda enviroment, which didn't use MKL, and use that python binary to work with julia.

Since parts of this solution are scattered everywhere and caused me a big head ache to put together, I though I would collect everything here:

Create Conda Enviorment without MKL

How to install scipy without mkl

I suggest in particular:

conda create -n pynomkl python nomkl

Then install the packages as usual

conda install -n pynomkl numpy pandas scipy scikit-learn ...

Doing this will create a special conda enviorment, and python binary that you can use just with julia's PyCall. That way you can continue to use MKL for other python only work.

find out where your python binary is:

> conda activate pynomkl                          (base) 
> python                                       (pynomkl)
>>> import sys
>>> sys.executable
'pathtopython/../python'

No fix PyCall: in julia:

ENV["PYTHON"]="pathtopython/../python"
using Pkg ; Pkg.build("PyCall")

Everything should work after that. If your julia pyimport code requires any libaries, make sure to install them in the pynomkl enviorment now.