shared libraries - python pip specify a library directory and an include directory

Question

I am using pip and trying to install a python module called pyodbc which has some dependencies on non-python libraries like unixodbc-dev, unixodbc-bin, unixodbc. I cannot install these dependencies system wide at the moment, as I am only playing, so I have installed them in a non-standard location. How do I tell pip where to look for these dependencies ? More exactly, how do I pass information through pip of include dirs (gcc -I) and library dirs (gcc -L -l) to be used when building the pyodbc extension ?

Answer 1

pip has a --global-option flag

You can use it to pass additional flags to build_ext.
For instance, to add a --library-dirs (-L) flag:
pip install --global-option=build_ext --global-option="-L/path/to/local" pyodbc

gcc supports also environment variables: http://gcc.gnu.org/onlinedocs/gcc/Environment-Variables.html

I couldn't find any build_ext documentation, so here is the command line help

Options for 'build_ext' command:
  --build-lib (-b)     directory for compiled extension modules
  --build-temp (-t)    directory for temporary files (build by-products)
  --plat-name (-p)     platform name to cross-compile for, if supported
                       (default: linux-x86_64)
  --inplace (-i)       ignore build-lib and put compiled extensions into the
                       source directory alongside your pure Python modules
  --include-dirs (-I)  list of directories to search for header files
                       (separated by ':')
  --define (-D)        C preprocessor macros to define
  --undef (-U)         C preprocessor macros to undefine
  --libraries (-l)     external C libraries to link with
  --library-dirs (-L)  directories to search for external C libraries
                       (separated by ':')
  --rpath (-R)         directories to search for shared C libraries at runtime
  --link-objects (-O)  extra explicit link objects to include in the link
  --debug (-g)         compile/link with debugging information
  --force (-f)         forcibly build everything (ignore file timestamps)
  --compiler (-c)      specify the compiler type
  --swig-cpp           make SWIG create C++ files (default is C)
  --swig-opts          list of SWIG command line options
  --swig               path to the SWIG executable
  --user               add user include, library and rpath
  --help-compiler      list available compilers

Answer 2

Building on Thorfin's answer and assuming that your desired include and library locations are in /usr/local, you can pass both in like so:

sudo pip install --global-option=build_ext --global-option="-I/usr/local/include/" --global-option="-L/usr/local/lib"  <you package name>

Answer 3

Just FYI... If you are having trouble installing a package with pip, then you can use the

--no-clean option to see what is exactly going on (that is, why the build did not work). For instance, if numpy is not installing properly, you could try

pip install --no-clean numpy

then look at the Temporary folder to see how far the build got. On a Windows machine, this should be located at something like:

C:\Users\Bob\AppData\Local\Temp\pip_build_Bob\numpy

Just to be clear, the --no-clean option tries to install the package, but does not clean up after itself, letting you see what pip was trying to do.

Otherwise, if you just want to download the source code, then I would use the -d flag. For instance, to download the Numpy source code .tar file to the current directory, use:

pip install -d %cd% numpy

Answer 4

Another way to indicate the location of include files and libraries are set relevant environment variables before running pip e.g.

export LDFLAGS=-L/usr/local/opt/openssl/lib
export CPPFLAGS=-I/usr/local/opt/openssl/include
pip install cryptography

Answer 5

I was also helped by Thorfin's answer; I was building GTK3+ on windows and installing pygobject, I was having difficulties on how to include multiple folders with pip install.

I tried creating pip config file as per pip documentation. but failed. the one working is with the command line:

pip install --global-option=build_ext --global-option="-IlistOfDirectories" 
# and/or with:  --global-option="-LlistofDirectories"

the separator that works with multiple folders in windows is ';' semicolon, NOT colon ':' it might be different in other OS.

sample working command line:

pip install --global-option=build_ext --global-option="-Ic:/gtk-build/gtk/x64/release/include;d:/gtk-build/gtk/x64/release/include/gobject-introspection-1.0" --global-option="-Lc:\gtk-build\gtk\x64\release\lib" pygobject==3.27.1

you can use '' or '/' for path, but make sure do not type backslash next to "

this below will fail because there is backslash next to double quote

pip install --global-option=build_ext --global-option="-Ic:\willFail\" --global-option="-Lc:\willFail\" pygobject==3.27.1

Answer 6

Have you ever used virtualenv? It's Python package that let's you create and maintain multiple isolated environments on one machine. Each can use different modules independent of one another without screwing up dependencies in your system library or a separate virtual environment.

If you don't have root privileges, you can download and use the virtualenv package from source:

$ curl -O https://pypi.python.org/packages/source/v/virtualenv/virtualenv-X.X.tar.gz
$ tar xvfz virtualenv-X.X.tar.gz
$ cd virtualenv-X.X
$ python virtualenv.py myVE

I followed the above steps this weekend on Ubuntu Server 12.0.4 and it worked perfectly. Each new virtual environment you create comes with PIP by default so installing packages into your new environment is easy.

Answer 7

Just in case it's of help to somebody, I still could not find a way to do it through pip, so ended up simply downloading the package and doing through its 'setup.py'. Also switched to what seems an easier to install API called 'pymssql'.