Trying to understand pytest-xdist failures for parametrized tests

Question

I have some parametrized tests that use a fixture defined in conftest. My tests are failing when pytest-xdist is used but pass with standard pytest. The fixture creates a directory which is then used by the test and I'm getting IOErrors.

From the output, it looks like the two failing parametrized tests are either being collected and/or being run multiple times. Note in the output below that test_save_load[obj2] and test_save_load[obj0] are failing and these appear twice so I assume they are being run multiple times. I can write the test differently to not parametrize; but was curious to understand this problem. Any ideas on what is going on or how pytest-xdist works to help understand the issue would be greatly appreciated.

Thanks!

Here's a snippet of my output:

platform darwin -- Python 2.7.6 -- pytest-2.5.1 -- 
plugins: xdist
[gw0] darwin Python 2.7.6 cwd: /tests/unit_tests
[gw1] darwin Python 2.7.6 cwd: /tests/unit_tests
[gw2] darwin Python 2.7.6 cwd: /tests/unit_tests
[gw3] darwin Python 2.7.6 cwd: /tests/unit_tests
[gw0] Python 2.7.6 (v2.7.6:3a1db0d2747e, Nov 10 2013, 00:42:54)  -- [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
[gw2] Python 2.7.6 (v2.7.6:3a1db0d2747e, Nov 10 2013, 00:42:54)  -- [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
[gw1] Python 2.7.6 (v2.7.6:3a1db0d2747e, Nov 10 2013, 00:42:54)  -- [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
[gw3] Python 2.7.6 (v2.7.6:3a1db0d2747e, Nov 10 2013, 00:42:54)  -- [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
gw0 [30] / gw1 [30] / gw2 [30] / gw3 [30]
scheduling tests via LoadScheduling

test_save_load.py:17: test_exceptions 
test_save_load.py:135: test_save_load[obj0] 
test_save_load.py:45: test_gnome_obj_reference 
test_save_load.py:135: test_save_load[obj2] 
[gw3] PASSED test_save_load.py:17: test_exceptions 
[gw1] PASSED test_save_load.py:45: test_gnome_obj_reference 
test_save_load.py:31: test_reference_object 
[gw3] PASSED test_save_load.py:31: test_reference_object 
test_save_load.py:73: test_savloc_created 
test_save_load.py:135: test_save_load[obj4] 
[gw1] PASSED test_save_load.py:73: test_savloc_created 
test_save_load.py:135: test_save_load[obj5] 
[gw3] PASSED test_save_load.py:135: test_save_load[obj4] 
test_save_load.py:135: test_save_load[obj6] 
[gw2] FAILED test_save_load.py:135: test_save_load[obj2] 
[gw0] FAILED test_save_load.py:135: test_save_load[obj0]

Bruno Oliveira Bruno Oliveira · Accepted Answer · 2015-02-14T13:11:11

xdist output is a little different than standard's pytest because the tests are running in parallel. When you see:

test_save_load.py:135: test_save_load[obj0]

it means that the test has been sent to one of the nodes for execution.

When you see:

[gw0] FAILED test_save_load.py:135: test_save_load[obj0]

it just means that the test has finished in node 0.

That being said, your tests are being executed only once. Your description of the test failure seems to indicate that the tests are sharing some disk resource (for instance, they are all trying to write to the same file name).

You should make sure that your tests that deal with files work in separate directories so each test is completely isolated from the others; you can accomplish that with the help of the tmpdir fixture.

If you post more information about the test themselves perhaps people can give more specific advice.

Cheers,

Trying to understand pytest-xdist failures for parametrized tests

1 Answers