Canary Builds and Tox

Recently I have been overhauling our CI configuration at work, so naturally I am keen to learn new CI techniques that I can integrate. Canary Builds piqued my interest:

Many projects have external code dependencies, a large amount of which is provided by open source projects. In order to ensure our builds are reproducible, we integrate against known versions of them, but that can mean that it takes a while for us to integrate against newer versions of these libraries leading to a larger merge effort down the line. One approach we have seen to avoid this is to have a nightly Canary Build which tries to pull in the latest version of all dependencies. If the build is green, we know we can change which versions we depend on. > - ThoughtWorks Technology Radar

An intriguing proposition; instead of tentatively nudging dependency versions, just let the CI server aggressively do it on a nightly basis and upgrade periodically when green.

So how easy is this to integrate in a Python environment?

Canary Builds with Tox

It is currently in the “Trial” ring of the radar, I am also keen to trial it using Tox.

Below is an example tox.ini:

[deps]
base =
    django-some-stable-lib>=1.4,<2.0
    django-shiny-cutting-edge-lib>=0.2,<0.3   
dj17 =
    Django>=1.7,<1.8       
py2 =
    # Python 2 specifics
py3 =
    # Python 3 specifics

envlist =
    py27-dj17,
    py34-dj17

[testenv]
commands=coverage run runtests.py

[testenv:py27-dj17]
basepython=python2.7
deps =
    {[deps]base}
    {[deps]py2}
    {[deps]dj17}

[testenv:py34-dj17]
basepython=python3.4
deps =
    {[deps]base}
    {[deps]py3}
    {[deps]dj17}

I have split the requirements into base and environment specifics like dj17, py2 and py3. I then have a testenv for each configuration that pulls in the correct dependencies.

Integration

I first of all add a canary dependency entry to deps, this is comprised of all the relevant dependencies without specifying any upper version limits:

[deps] 
base =
    #...

canary =
    # Take the newest of everything.
    Django>=1.7
    django-some-stable-lib>=1.4
    django-shiny-cutting-edge-lib>=0.2

dj17 = 
    #...

Next I add a canary environment entry to the envlist:

envlist =
    py27-dj17,
    py34-dj17,
    canary

And finally the canary testenv:

[testenv:canary]
pip_pre=True
basepython=python3.4
deps =
    {[deps]canary}

I use pip_pre=True here so tox passes the --pre argument to pip, allowing pip to install non-final (alpha/beta/release candidate) packages.

I have decided for now to just have a canary build for Python 3.4 as it is the latest python we support.

All I need to do is add a nightly scheduled build to my CI server to execute the build by calling tox -r -e canary (-r to recreate the virtualenv and -e to specify the environment).

Programming in your CI/CD

As an aside; Tox is an excellent example of the negatives highlighted by the On Hold blip: Programming in your CI/CD.

We still see teams configure their CI and CD tools by directly embedding complex multi-line commands into the configuration of the tool. Often these embedded commands also contain steps that would only ever take effect in the build environment including things such as CI specific environment variables, steps that would create/modify files and templates only in the CI environment etc. This makes the build environment a special beast - whose results cannot be duplicated locally on a developer’s machine. > - ThoughtWorks Technology Radar

By relying on Tox to run our tests everywhere, we avoid any CI complex build scripts and special cases whilst also making it easy to run and debug any environment configuration on developer machines.

I recommend that you checkout the ThoughtWorks Technology Radar and see if you find anything to add to your own radar.