How to compile and link multiple python modules (or packages) using cython?

20
votes

I have several python modules (organized into packages), which depend on each other. e.g.

  • Module1
  • Module2: imports Module1
  • Module3
  • Module4: imports Module3, Module 2, Module 1

Let's assume the relevant interface to develop applications is in Module4 and I want to generate a Module4.so using cython. If I proceed in the naive way, I get an extension Module4.so which I can import BUT the extension relies on the python source code of Module1, Module2, Module3.

Is there a way to compile so that also Module1,Module2, Module3 are compiled and linked to Module4? I would like to avoid doing everything manually, e.g. first compile Module1.so then change import declaration in Module2, so as to import Module1.so rather than Module1.py, then compile Module2 into Module2.so and so on....

Thanks!

1
pythoncompilationpackagecython

1 Answers

12
votes

Edit. First two options refer to Cython's specific code, what I've missed is that the question is about pure python modules, so option 3 is the solution.

There are a few options:

1. See this "How To Create A Hierarchy Of Modules In A Package": https://github.com/cython/cython/wiki/PackageHierarchy

2. I prefer the "include" statement: http://docs.cython.org/src/userguide/language_basics.html#the-include-statement I have many .pyx files and they are all included in main.pyx, it's all in one namespace. The result is one big module: http://code.google.com/p/cefpython/source/browse/cefpython.pyx

3. You can compile all your modules at once using setup by adding more than one "Extension":

setup(
    cmdclass = {'build_ext': build_ext},
    ext_modules = [Extension("example", sourcefiles), Extension("example2", sourcefiles2), Extension("example3", sourcefiles3)]
)

4. A more efficent compilation - see here.

setup (
    name = 'MyProject',
    ext_modules = cythonize(["*.pyx"]),
)