Home / Resolve cython and numpy dependencies on setup step

Resolve cython and numpy dependencies on setup step

Resolve cython and numpy dependencies on setup step

I have a custom Python package on my project where some functionality implemented on Cython. We use it to speed up some of the slowest parts of our code - the syntax is pretty close (it can be identical except some imports) to regular Python. Cython compiled to .so libraries and it has support for fast access to NumPy arrays.

The problem - Cython and numpy packages need to be installed before the actual setup starts its work.

The setup.py look as follows:

import numpy
from Cython.Build import cythonize

setup(
    name=...,
    version=...,
    ...
    ext_modules=cythonize([
        Extension('package.cython_code1', ['package/cython_code1.pyx']),
        Extension('package.cython_code1', ['package/cython_code2.pyx']),
    ]),
    include_dirs=[numpy.get_include()]
)

There several ways to handle Cython and numpy :

First:

Starting from 18.0 release of setuptools (released on 2015-06-23) it is possible to specify Cython in setup_requires and pass .pyx modules sources for regular setuptools.Extension:

from setuptools import setup, Extension

setup(
    ...
    setup_requires=[
        # Setuptools 18.0 properly handles Cython extensions.
        'setuptools>=18.0',
        'cython',
    ],
    ext_modules=[
        Extension('package.cython_code1', sources=['package/cython_code1.pyx']),
        Extension('package.cython_code2', sources=['package/cython_code2.pyx']),
    ],
)

If your pip is older or you don't know it's version if you put cython dependency into setup_requires it will fail with ImportError like:

ImportError: No module named Cython.Build

In order to fix it you can wrap your cythonize function into dummy closure:

try:
    from Cython.Build import cythonize
except ImportError:
    # create closure for deferred import
    def cythonize (*args, ** kwargs ):
        from Cython.Build import cythonize
        return cythonize(*args, ** kwargs)

Then later when the setup_requires argument is handled, Cython will be installed and the setup script will be re-executed. Since at that point Cython is installed, you'll be able to successfully import cythonize

Or stick with version from https://github.com/pypa/pip/issues/5761. Where you install cython and numpy using setuptools.dist before actual setup:

from setuptools import dist
dist.Distribution().fetch_build_eggs(['Cython>=0.15.1', 'numpy>=1.10'])
Support author