py.test 2.3.5 does not run finalizer after fixture failure

Question

i was trying py.test for its claimed better support than unittest for module and session fixtures, but i stumbled on a, at least for me, bizarre behavior.

Consider the following code (don't tell me it's dumb, i know it, it's just a quick and dirty hack to replicate the behavior) (i'm running on Python 2.7.5 x86 on windows 7) import os import shutil import pytest

test_work_dir = 'test-work-dir'
tmp = os.environ['tmp']
count = 0

@pytest.fixture(scope='module')
def work_path(request):
    global count
    count += 1
    print('test: ' + str(count))
    test_work_path = os.path.join(tmp, test_work_dir)

    def cleanup():
        print('cleanup: ' + str(count))
        if os.path.isdir(test_work_path):
            shutil.rmtree(test_work_path)
    request.addfinalizer(cleanup)
    os.makedirs(test_work_path)
    return test_work_path

def test_1(work_path):
    assert os.path.isdir(work_path)
def test_2(work_path):
    assert os.path.isdir(work_path)
def test_3(work_path):
    assert os.path.isdir(work_path)

if __name__ == "__main__":
    pytest.main(['-s', '-v', __file__])

If test_work_dir does not exist, then i obtain the expected behavior:

platform win32 -- Python 2.7.5 -- pytest-2.3.5 -- C:\Programs\Python\27-envs\common\Scripts\python.exe
collecting ... collected 4 items

py_test.py: [doctest] PASSED
py_test.py:34: test_1 test: 1
cleanup: 1
PASSED
py_test.py:38: test_2 PASSED
py_test.py:42: test_3 PASSEDcleanup: 1

fixture is called once for the module and cleanup is called once at the end of tests.

Then if test_work_dir exists i would expect something similar to unittest, that fixture is called once, it fails with OSError, tests that need it are not run, cleanup is called once and world peace is established again.

But... here's what i see:

py_test.py: [doctest] PASSED
py_test.py:34: test_1 test: 1
ERROR
py_test.py:38: test_2 test: 2
ERROR
py_test.py:42: test_3 test: 3
ERROR

Despite the failure of the fixture all the tests are run, the fixture that is supposed to be scope='module' is called once for each test and finalizer in never called!

I know that exceptions in fixtures are not good policy, but the real fixtures are complex and i'd rather avoid filling them with try blocks if i can count on the execution of each finalizer set till the point of failure. I don't want to go hunting for test artifacts after a failure. And moreover trying to run the tests when not all of the fixtures they need are in place makes no sense and can make them at best erratic.

Is this the intended behavior of py.test in case of failure in a fixture?

Thanks, Gabriele

Hi Gabriele. I hate to say this, but for me it works. Is your example complete, maybe there is something missing? — Kanguros
Do you mean that with pytest 2.3.5 after a fixture failure you are able to obtain the scheduling of the finalizer? — Gabriele

hpk42 hpk42 · Accepted Answer · 2013-07-17T08:35:12

Three issues here:

you should register the finalizer after you performed the action that you want to be undone. So first call makedirs() and then register the finalizer. That's a general issue with fixtures because usually teardown code can only run if there was something successfully created
pytest-2.3.5 has a bug in that it will not call finalizers if the fixture function fails. I've just fixed it and you can install the 2.4.0.dev7 (or higher) version with pip install -i http://pypi.testrun.org -U pytest. It ensures the fixture finalizers are called even if the fixture function partially fails. Actually a bit surprising this hasn't popped up earlier but i guess people, including me, usually just go ahead and fix the fixtures instead of diving into what's happening specifically. So thanks for posting here!
if a module-scoped fixture function fails, the next test needing that fixture will still trigger execution of the fixture function again, as it might have been an intermittent failure. It stands to reason that pytest should memorize the failure for the given scope and not retry execution. If you think so, please open an issue, linking to this stackoverflow discussion.

thanks, holger

py.test 2.3.5 does not run finalizer after fixture failure

1 Answers