Friday, 7 June 2019

Is it possible to test that test-dependencies have not leaked into real code with python/pytest

TL;DR: If I'm using pytest and some other test-only dependencies, is it possible to assert that none of these test-only dependencies have leaked into actual, non-test code?

In Java, when I run tests, the tests themselves and the code-under-test will run in different classloaders, with their own dependencies scoped there. So if the code-under-test inadvertently referenced testng, the tests will fail, even though testng is running the tests.

Can the same be achieved in Python? If my non-test code accidentally imports pytest, can I catch that and have the tests fail? I can't see how to achieve that.

Although setuptools and pip etc. make it relatively easy to keep install/run and dev/test dependencies separate, and even avoid polluting the current venv, they are still present when the tests run. This means that running python setup.py test for a module can pass, but python setup.py install followed by something as simple as importing the module can instafail.

Given: cat setup.py

from setuptools import setup, find_packages

setup(
    name="brettpy",
    version="0.0.1",
    packages=find_packages(),
    setup_requires=["pytest-runner",],
    tests_require=["pytest",],
)

cat setup.cfg

[aliases]
test=pytest

cat brettpy/__init__.py

import pytest

cat tests/test_brettpy.py

def test_import_brettpy():
    import brettpy
    del brettpy

def test_run_main():
    from brettpy import __main__ as main
    main.main()

... python setup.py test will pass, but python setup.py install && python -m brettpy will fail with:

ModuleNotFoundError: No module named 'pytest'

How do others ensure test-dependencies don't bleed into real code and cause missing-dependencies bugs when installed? Feels like a bug that the test-framework should be able to catch.



from Is it possible to test that test-dependencies have not leaked into real code with python/pytest

No comments:

Post a Comment