How to identify compiler before defining cython extensions?

7
votes

I'm building a platform independent cython project where I want to pass compiler args based on the compiler being used. I can guess the compiler based on platform or assume it's the same compiler used for Python but it's not guaranteed to match. Normally I inject into the cmdclass arg to setuptool's setup method and wrap the install or build_ext commands to check internal state. But in this case I have to cythonize the extension modules before I reach the wrappers.

Is there any way to determine the compiler inside setup.py before cythonizing the extension modules?

3
cythonsetuptools
can't you pass the compiler as argument to setup.py: python setup.py build --compiler=mingw32? - denfromufa
you can also use cmake to compile cython code in cross-platform manner: github.com/thewtex/cython-cmake-example - denfromufa
@denfromufa You can pass --compiler=mingw32, but other recipients of the repository won't necessarily know what to set the compiler argument to, or if it's a dependency of another repo. And pip install definitely won't create such an argument to setuptools. I could read the argument if i were only using python setup.py install myself -- that is true. - Pyrce
Also I hadn't seen cython-cmake before -- I'll look into that. I'd rather not introduce a complicated CMake pattern into the repository if I can avoid it and would like to instead just set the appropriate compiler flags in setuptools based on the compiler that's been picked (by user or setuptools or python). But it appears to allow for compiler specific arguments after cythonization is defined. - Pyrce

3 Answers

4
votes

After posting on the cython forums and searching for related issues in distutils I found this post showing how to move the compiler arguments into the build_ext assignment. If I subsequently remove all compiler arguments from the extension class I can now lazy assign them inside the command class as I expected. I can also get install and egg_info command classes to call my new version of the build_ext as well.

from setuptools.command.build_ext import build_ext

BUILD_ARGS = defaultdict(lambda: ['-O3', '-g0'])
for compiler, args in [
        ('msvc', ['/EHsc', '/DHUNSPELL_STATIC']),
        ('gcc', ['-O3', '-g0'])]:
    BUILD_ARGS[compiler] = args

class build_ext_compiler_check(build_ext):
    def build_extensions(self):
        compiler = self.compiler.compiler_type
        args = BUILD_ARGS[compiler]
        for ext in self.extensions:
            ext.extra_compile_args = args
        build_ext.build_extensions(self)

...
setup(
    ...
    cmdclass={ 'build_ext': build_ext_compiler_check })
1
votes

A simple variation of the first answer:

from setuptools import setup, Extension
from distutils.command.build_ext import build_ext

myextension = Extension(
    name = 'packagename',
    sources = [
        'source/debugger.cpp',
    ],
    include_dirs = [ 'source' ],
)

class build_ext_compiler_check(build_ext):
    def build_extensions(self):
        compiler = self.compiler.compiler_type

        # print('\n\ncompiler', compiler)
        if not 'msvc' in compiler:

            for extension in self.extensions:

                if extension == myextension:
                    extension.extra_compile_args.append( '-O0' )
                    extension.extra_compile_args.append( '-std=c++11' )

        super().build_extensions()

setup(
        name = 'packagename',
        version = __version__,
        ext_modules= [ myextension ],
    )
0
votes

(Sorry can't comment due to missing credit)

Unfortunately, the answer https://stackoverflow.com/a/32192172/7379507 is slightly misleading as a build_ext instance's self.compiler.compiler_type is 'unix', not 'gcc' (the "distutils compiler_class").

I.e. the lookup from this defaultdict dictionary

BUILD_ARGS = defaultdict(lambda: ['-O3', '-g0'])
for compiler, args in [
    ('msvc', ['/EHsc', '/DHUNSPELL_STATIC']),
    ('gcc', ['-O3', '-g0'])]:
BUILD_ARGS[compiler] = args

wouldn't normally reach the 'gcc' entry but instead always fall back to the defaultdict's default (the lambda function).

That said you probably wouldn't notice this in a majority of cases as long as the default options remain the same as the 'gcc' options. E.g. clang seems to understand the same options as gcc.

It looks like you might be able to get at the actual invoked compiler name through self.compiler.compiler[0], though I haven't checked if this is reliable or portable.