Resolve cython and numpy dependencies on setup step

In my project, I have a custom Python package that has part of its functionality implemented on Cython. We use it to accelerate some of the slowest parts of our code — the syntax is pretty close to regular Python (it can be absolutely identical except for some imports). Cython compiled with .so libraries can directly access low-level arrays of numpy.

The problem — Cython and numpy packages need to be installed before the setup.py starts its work.

The setup.py look as follows:

import numpy
from Cython.Build import cythonize

setup(
    name=...,
    version=...,
    ...
    ext_modules=cythonize([
        Extension('package.cython_code1', ['package/cython_code1.pyx']),
        Extension('package.cython_code1', ['package/cython_code2.pyx']),
    ]),
    include_dirs=[numpy.get_include()],
)

There several ways to handle Cython and numpy:

First:

Starting from 18.0 release of setuptools (released on 2015-06-23) it is possible to specify Cython in setup_requires and pass .pyx modules sources for regular setuptools.Extension:

from setuptools import setup, Extension

setup(
    ...
    setup_requires=[
        # Setuptools 18.0 properly handles Cython extensions.
        'setuptools>=18.0',
        'cython',
    ],
    ext_modules=[
        Extension('package.cython_code1', sources=['package/cython_code1.pyx']),
        Extension('package.cython_code2', sources=['package/cython_code2.pyx']),
    ],
    include_dirs=[numpy.get_include()],
)

If your installation tool is older and you have put the Cython dependency in setup_requires, it will not work with ImportError like:

ImportError: No module named Cython.Build

In order to fix this, you can wrap the cythonize function in a dumb closure:

try:
    from Cython.Build import cythonize
except ImportError:
    # create closure for deferred import
    def cythonize (*args, ** kwargs ):
        from Cython.Build import cythonize
        return cythonize(*args, ** kwargs)

Then, when the setup_requires argument is processed, Cython will be installed and the installation script will continue. Since Cython is installed at this point, you can successfully import and use cythonize.

Or you can stick to https://github.com/pypa/pip/issues/5761. Here you install cython and numpy using setuptools.dist before actual setup:

from setuptools import dist
dist.Distribution().fetch_build_eggs(['Cython>=0.15.1', 'numpy>=1.10'])