Using pytest as test runner while preventing it from modifying test code

Question

Is there a way to use pytest as a test runner to collect some unit tests but prevent pytest from modifying the test code itself on selected unit tests?

As part of a series of unit tests in Python, I have some that use functions that rely on frame inspection. If I run these on their own, they produce the expected results. However, if I use pytest to collect the tests and run them, the tests fail as pytest inserts additional frames which end up generating a RuntimeError.

Here's a sample file that works when run on its own using only Python. I added a print statement so as to show one of the frames added by pytest before an exception was raised.

import inspect


global_a = 1


def get_nonlocals(frame, print_frame=False):
    """Returns a list of variables in a nonlocal scope, whether they
    were declared as such or not.
    """
    globals_ = frame.f_globals

    nonlocals_ = {}
    while frame.f_back is not None:
        if print_frame:
            print(frame)
        frame = frame.f_back
        for key in frame.f_locals:
            if key in globals_ or key in nonlocals_:
                continue
            nonlocals_[key] = frame.f_locals[key]
    return nonlocals_


def test_get_nonlocals():
    # We cannot use pytest for this test as it messes with the frames
    # and generates a RuntimeError.

    current_frame = None
    nonlocal_b = 2

    def outer():
        nonlocal_c = 3
        def inner():
            nonlocal current_frame
            current_frame = inspect.currentframe()
        inner()

    outer()


    assert "global_a" not in get_nonlocals(current_frame, print_frame=True)

    for var in ["nonlocal_b", "nonlocal_c", "current_frame"]:
        assert var in get_nonlocals(current_frame)



if __name__ == '__main__':
    test_get_nonlocals()

First, the result of simply using "python test.py"; no exception is raised.

<frame at 0x015C98F8, file 'test.py', line 38, code inner>
<frame at 0x015FB1A8, file 'test.py', line 39, code outer>
<frame at 0x015C91A0, file 'test.py', line 44, code test_get_nonlocals>

Next, running the same file with pytest test.py

-------------------- Captured stdout call ---------------
<frame at 0x039B12C8, file '...test.py', line 38, code inner>
<frame at 0x039B0838, file '...test.py', line 39, code outer>
<frame at 0x0396C4F0, file '...test.py', line 44, code test_get_nonlocals>
<frame at 0x039AE868, file '...\\lib\\site-packages\\_pytest\\python.py', line 184, code pytest_pyfunc_call>
============================= short test summary info =================
FAILED test.py::test_get_nonlocals - RuntimeError: dictionary keys changed during iteration

André André · Accepted Answer · 2020-10-17T13:02:36

After filing an issue with pytest, I got a way to get the test pass with pytest, by replacing

for key in frame.f_locals:

by

for key in list(frame.f_locals):

No additional information was given as to why this works exactly.

Using pytest as test runner while preventing it from modifying test code

1 Answers