How to determine which compiler will be called to extend Python C in setuptools?

Question

I have a Python C ++ extension that requires the following compilation flags when compiling using clang on OSX:

CPPFLAGS='-std=c++11 -stdlib=libc++ -mmacosx-version-min=10.8' LDFLAGS='-lc++' 

Finding OSX in my setup.py file is quite simple. I can do it:

 if sys.prefix == 'darwin': compile_args.append(['-mmacosx-version-min=10.8', '-stdlib=libc++']) link_args.append('-lc++') 

(See https://github.com/honnibal/spaCy/blob/ba1d3ddd7f527d2e6e41b86da0f2887cc4dec83a/setup.py#L70 for full context)

However, in GCC, this compilation flag is invalid. Thus, compilation will fail if someone tries to use GCC on OSX, if I write setup.py in this way.

GCC and clang support different compiler flags. So, I need to know which compiler will be called so that I can send various flags. How to correctly determine the compiler in the setup.py file?

Change 1:

Note that the Python exception does not occur for compilation errors:

 $ python setup.py build_ext --inplace running build_ext building 'spacy.strings' extension gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -c spacy/strings.cpp -o build/temp.linux-x86_64-2.7/spacy/strings.o -O3 -mmacosx-version-min=10.8 -stdlib=libc++ gcc: error: unrecognized command line option '-mmacosx-version-min=10.8 gcc: error: unrecognized command line option '-stdlib=libc++ error: command 'gcc' failed with exit status 1 $ 

Answer 1

I pressed your question since I need the same switch. Also, in my case, sys.prefix is small because the flags for clang are platform independent.

I'm not sure this is great, but here is what works best for me. So, I check if the CC variable is set; if not, I check that I think where distutils looks.

Any best solution is welcome!

 import os import distutils try: if os.environ['CC'] == "clang": clang = True except KeyError: clang = False if clang or distutils.sysconfig_get_config_vars()['CC'] == 'clang': try: _ = os.environ['CFLAGS'] except KeyError: os.environ['CFLAGS'] = "" os.environ['CFLAGS'] += " -Wno-unused-function" os.environ['CFLAGS'] += " -Wno-int-conversion" os.environ['CFLAGS'] += " -Wno-incompatible-pointer-types 

Note for rude guys: I would love to use the extra_compile_args parameter, but it puts the flags in the wrong position on the clang compilation command.

Answer 2

Add the following code to your setup.py . It explicitly determines which flags are accepted by the compiler, and then only those are added.

 # check whether compiler supports a flag def has_flag(compiler, flagname): import tempfile from distutils.errors import CompileError with tempfile.NamedTemporaryFile('w', suffix='.cpp') as f: f.write('int main (int argc, char **argv) { return 0; }') try: compiler.compile([f.name], extra_postargs=[flagname]) except CompileError: return False return True # filter flags, returns list of accepted flags def flag_filter(compiler, *flags): result = [] for flag in flags: if has_flag(compiler, flag): result.append(flag) return result class BuildExt(build_ext): # these flags are not checked and always added compile_flags = {"msvc": ['/EHsc'], "unix": ["-std=c++11"]} def build_extensions(self): ct = self.compiler.compiler_type opts = self.compile_flags.get(ct, []) if ct == 'unix': # only add flags which pass the flag_filter opts += flag_filter(self.compiler, '-fvisibility=hidden', '-stdlib=libc++', '-std=c++14') for ext in self.extensions: ext.extra_compile_args = opts build_ext.build_extensions(self) setup( cmdclass=dict(build_ext=BuildExt), # other options... ) 

The has_flag method was taken from this pybind11 example. https://github.com/pybind/python_example