37
votes

Pytest test discovery is failing. The UI states:

Test discovery error, please check the configuration settings for the tests

The output window states:

Test Discovery failed: 
Error: Traceback (most recent call last):
  File "C:\Users\mikep\.vscode\extensions\ms-python.python-2019.4.11987\pythonFiles\testing_tools\run_adapter.py", line 16, in <module>
    main(tool, cmd, subargs, toolargs)
  File "C:\Users\mikep\.vscode\extensions\ms-python.python-2019.4.11987\pythonFiles\testing_tools\adapter\__main__.py", line 90, in main
    parents, result = run(toolargs, **subargs)
  File "C:\Users\mikep\.vscode\extensions\ms-python.python-2019.4.11987\pythonFiles\testing_tools\adapter\pytest.py", line 43, in discover
    raise Exception('pytest discovery failed (exit code {})'.format(ec))
Exception: pytest discovery failed (exit code 3)

Here are my settings:

{
    "python.pythonPath": ".venv\\Scripts\\python.exe",
    "python.testing.pyTestArgs": [
        "tests"
    ],
    "python.testing.unittestEnabled": false,
    "python.testing.nosetestsEnabled": false,
    "python.testing.pyTestEnabled": true
}

I can run pytest from the command line successfully FWIW.

14
How are you running pytest from the command-line? Are you using pytest from your virtual environment with the same args as specified in your settings.json? - Brett Cannon
From the command line I'm just doing pytest tests from within the same virtual env that VS Code is using. - devlife
yea I also run pytest from the terminal within VSCode and than it was discovered, so that I could run it from the GUI afterwards. - Martin Alexandersson

14 Answers

7
votes

I resolved the issue by upgrading pytest to the latest version: 4.4.1 with "pip install --upgrade pytest". I was apparently running an old version 3.4.2

58
votes

I spent ages trying to decipher this unhelpful error after creating a test that had import errors. Verify that your test suite can actually be executed before doing any deeper troubleshooting.

pytest --collect-only is your friend.

24
votes

This is not a complete answer as I do not know why this is happening and may not relate to your problem, depending how you have your tests structured.

I resolved this issue by putting an __init__.py file in my tests folder

E.G.:


├───.vscode
│       settings.json
│
├───app
│       myapp.py
│
└───tests
        test_myapp.py
        __init__.py

this was working a few days ago without this but the python extension was recently updated. I am not sure if this is the intended behavior or a side effect of how discoveries are now being made

https://github.com/Microsoft/vscode-python/blob/master/CHANGELOG.md
Use Python code for discovery of tests when using pytest. (#4795)

11
votes

I just thought I would add my answer here as this might well affect someone who uses a .env file for their project's environment settings since it is such a common configuration for 12 factor apps.

My example assumes that you're using pipenv for your virtual environment management and that you have a .env file at the project's root directory.

My vscode workspace settings json file looks like below. The crucial line for me here was "python.envFile": "${workspaceFolder}/.env",

{
    "python.pythonPath": ".venv/bin/python",
    "python.linting.enabled": true,
    "python.linting.pylintEnabled": true,
    "python.linting.pycodestyleEnabled": false,
    "python.linting.flake8Enabled": false,
    "python.linting.pylintPath": ".venv/bin/pylint",
    "python.linting.pylintArgs": [
        "--load-plugins=pylint_django",
    ],
    "python.formatting.provider": "black",
    "python.formatting.blackArgs": [
        "--line-length",
        "100"
    ],
    "python.testing.unittestEnabled": false,
    "python.testing.nosetestsEnabled": false,
    "python.testing.pytestEnabled": true,
    "python.testing.pytestPath": ".venv/bin/pytest",
    "python.envFile": "${workspaceFolder}/.env",
    "python.testing.pytestArgs": [
        "--no-cov"
    ],
}

I hope this saves someone the time I spent figuring this out.

8
votes

Seems like a bug in the latest version of VS Code Python extension. I had the same issue, then I downgraded the Python extension to 2019.3.6558 and then it works again. So we should go to our VS Code extensions list, select the Python extension and "Install another version..." from the setting of that extension.

I hope this works for you too.

6
votes

In my case the problem with vscode being unable to discover tests was the coverage module being enabled in the setup.cfg (alternatively this could be a pytest.ini), i.e.

addopts= --cov <path> -ra

which caused the test discovery to fail due to low coverage. The solution was to remove that line from the config file.

Also, as suggested in the documentation:

You can also configure testing manually by setting one and only one of the following settings to true: python.testing.unittestEnabled, python.testing.pytestEnabled, and python.testing.nosetestsEnabled.

EDIT:

Slightly related - in case of more complex projects it might be also necessary to change the rootdir parameter (inside your settings.json) when running tests with pytest (in case of ModuleNotFoundError):

    "python.testing.pytestArgs": [
        "--rootdir","${workspaceFolder}/<path-to-directory-with-tests>"
    ],
3
votes

Before begin the tests discovery, check that python.testing.cwd points correctly to your tests dir and your python.testing.pytestEnabled is set to true.

Once those requirements are set correctly, run tests discovery and its output (see OUTPUT window). You should see something like this:

python $HOME/.vscode/extensions/ms-python.python-$VERSION/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir $ROOT_DIR_OF_YOUR_PROJECT -s --cache-clear
Test Discovery failed: 
Error: ============================= test session starts ==============================
<SETTINGS RELATED TO YOUR MACHINE, PYTHON VERSION, PYTEST VERSION,...>
collected N items / M errors
...

It's important to highlight here the last line: collected N items / M errors. The following lines will contain info about the tests discovered by pytest. So your tests are discoverable but there are errors related to their correct execution.

The following lines will contain the errors which in most of the cases will be related to an incorrect import.

Check that all your dependencies have been downloaded previously. If you are working with specific versions of dependencies (on therequirements.txt file you have something like your_package == X.Y.Z) be sure that it's the right version that you need to.

2
votes

If you are having trouble with pytest I was struggling with the discovery test part.

Reading some open issue (other) in vscode I found a workaround using the test-adapter extension

market_place_link

The extention works like a charm. (and solve my discovery problem)

1
votes

This error is so frustrating.. In my case the error fixed by modifying the python.pythonPath parameter in settings.json (found inside .vscode folder under project root directory), to the path which I obtained using which python in terminal (e.g. /usr/local/var/pyenv/shims/python)

I use pyenv with python3.9, and my error said previously:

Error: Process returned an error: /Users/"user-name"/.pyenv/shims/python: line 21: /usr/local/Cellar/pyenv/1.2.23/libexec/pyenv: No such file or directory

at ChildProcess. (/Users/"user-name"/.vscode/extensions/littlefoxteam.vscode-python-test-adapter-0.6.8/out/src/processRunner.js:35:36) at Object.onceWrapper (events.js:422:26) at ChildProcess.emit (events.js:315:20) at maybeClose (internal/child_process.js:1021:16) at Process.ChildProcess._handle.onexit (internal/child_process.js:286:5)

0
votes

Looking at https://docs.pytest.org/en/latest/usage.html#possible-exit-codes it seems pytest itself is falling over do to some internal error.

0
votes

I had this same issue and tracked it down to my pytest.ini file. Removing most of the addopts from that file fixed the issue.

0
votes

I searched in the settings for "python" and found this:

enter image description here

Switching to pytest automatically detect my tests:

enter image description here

0
votes

I had this problem and struggle with it for hours. I think that there is a special resolution for every other platform configuration. My platform is:

  • VSCode: 1.55.0 (Ubuntu)
  • Pytest: 6.2.3
  • MS Python extension (ms-python.python 2021.3.680753044)
  • Python Test Explorer for Visual Studio Code (littlefoxteam.vscode-python-test-adapter - 0.6.7)

The worst thing has that the tool itself does not have a standard output (at least that I know of or could find easily on the internet).

In the end, the problem was

--no-cov

parameters that was not recognized by the VSCode testing explorer tool (that I copied from some page on the internet) and the error was showed by the extension littlefoxteam.vscode-python-test-adapter and it may help you to find where things are broken.

0
votes

In my case, same problem appeared each time when flake8 linter reported errors. Even one error was enough to fail VS Code test discovery. So the fix is either disable the linter or fix linter errors. I use setup described here.