I am using Cython as part of my build setup for a large project, driven by CMake. I can't seem to get Cython to generate the .c files in a sensible location.
My file layout:
C:\mypath\src\demo.py # Cython source file
C:\mypath\build\bin # I want demo.pyd to end up here
C:\mypath\build\projects\cyt\setup.py # Generated by CMake
My setup.py is generated by CMake (a lot of it depends on configure_file), in the location specified above. This location conforms to the usual structure of the overarching project (which builds over a hundred libraries and executables) and is not something I want to (or can easily) change.
The generated setup.py looks like this:
from distutils.core import setup, Extension
from Cython.Build import cythonize
import os.path
extension_args = {
'extra_compile_args' : ['/DWIN32','/DWIN64'],
'extra_link_args' : ['/MACHINE:X64'],
}
source = '../../../src/demo.py'
modules = [Extension(
os.path.splitext(os.path.basename(source))[0],
sources = [source],
**extension_args
)]
modules = cythonize(
modules,
build_dir = 'BUILD_DIR',
compiler_directives = {'language_level' : 2}
)
setup(name = 'demo',
version = '0.1',
description = '',
ext_modules = modules)
(Note that this is heavily simplified compared to the real case, which passes many additional arguments in extension_args, and includes many source files, each with its own object in modules. Nevertheless, I have verified that the minimised version above reproduces my issue).
Cython is run like this:
cd C:\mypath\build\projects\cyt
python setup.py build_ext --build-lib C:/mypath/build/bin --build-temp C:/mypath/build/projects/cyt
Ideally, I would want all intermediary build artefacts from Cython (the generated C files, object files, exp files, etc.) to reside somewhere in or below C:\mypath\build\projects\cyt. However, I can't seem to achieve that. Here is where build artefacts actually end up:
demo.pydends up inC:\mypath\build\bin, where I want it. No problem here.- The object file
demo.obj, along with the linked filesdemo.expanddemo.lib, end up inC:\mypath\build\projects\src. I want them insidecyt. - The C file
demo.cends up inC:\mypath\build\src. Again, I want this inprojects\cyt.
In the setup.py, I am setting the build_dir parameter for cythonize as suggested in this answer, but it doesn't seem to work as I would like. I also tried using cython_c_in_temp as per another answer on that question, but that has no effect (and judging from my inspection of Cython source code, does not apply to cythonize calls at all).
I tried using an absolute paths for source, but that made things even worse, as the C file ended up generated right next to demo.py, inside the source tree (as C:\src\demo.c).
My question: How can I make sure that all the generated intermediary files (C, obj, and friends) end up in the same directory as the generated setup.py, or below that?
I can think of two workarounds for my situation, but they both feel like hacks which I would like to avoid:
- Copy all the Python source files from their locations in
C:\mypath\srcto alongside the generatedsetup.py, so that I can refer to them without..in the path. That would likely solve the issue, but burdens the (already long) build process with tens of additional file copy operations I'd rather avoid. - Since the path where the files end up seems to be composed by concatenating "the directory of
setup.py+ the value ofbuild_dir+ the value ofsource", I could count the number of..in thesourcepath and specifybuild_dirdeep enough so that the evaluation results in the path I actually want. This is both extremely hacky and very fragile.
I hope a better solution exists.
from Can I achieve precise control over location of .c files generated by cythonize?
No comments:
Post a Comment