Azure DevOps can't find python module

Question

I am attempting to run tests from a python module using Azure DevOps. I've got a pipeline build set up to build off a yml file. I'm getting an error that my module name on my imports is not right, but it doesn't look like that the module name is getting set in the build?

When I run this locally, it works just fine.

Module repo file structure:

api_check
api_check/auth
api_check/etc
api_check/__init__.py

I'm running the tests using the following in my yml file: - script: | mkdir logs pytest -m smoke --junitxml=logs/test-results.xml --ENV=prod displayName: 'Test with pytest'

This step is giving the following error:

2018-10-11T21:51:02.9825682Z _______________________ ERROR collecting test_search.py ________________________
2018-10-11T21:51:02.9826322Z ImportError while importing test module '/home/vsts/work/1/s/test_search.py'.
2018-10-11T21:51:02.9826804Z Hint: make sure your test modules/packages have valid Python names.
2018-10-11T21:51:02.9827123Z Traceback:
2018-10-11T21:51:02.9827411Z test_search.py:12: in <module>
2018-10-11T21:51:02.9827745Z     from api_check.auth import get_Token_UsernamePassword
2018-10-11T21:51:02.9828644Z E   ModuleNotFoundError: No module named 'api_check'

Is there something I need to add to get the build to recognize the module api_check? Is there some sort of build variable I need to read in?

Edit for more information-

I am running this from whatever the directory is that the yml file gets run from my default - based off the debug log, the working directory is /home/vsts/work/1/s. When I list the contents of the working directory, I see all the contents of my module, everything that in my repo is under api_check.

So, rather than what I would expect from pulling down a repo:

$workingDirectory/api_check
$workingDirectory/api_check/auth
$workingDirectory/api_check/etc
$workingDirectory/api_check/__init__.py

it is instead:

$workingDirectory
$workingDirectory/auth
$workingDirectory/etc
$workingDirectory/__init__.py

So it looks like just adding the parent directory api_check wouldn't help, as there appears to be no api_check directory. Is there some setting I can flip in the pipeline sources to have it include the top-level folder api_check?

The parent dir of api_check is not in sys.path. However, it's not clear from where you are running pytest to elaborate a proper solution; adding path of api_check parent dir to PYTHONPATH should be the quick-and-dirty workaround. — hoefling
@hoefling I added further information in the main post about the directory situation. — klreeher
So api_check is the git repo root? Check whether you can modify the git clone command in your Azure container (e.g. git clone https://my.repo.git -o api_check) to match the local directory structure. However, this is only a workaround; modifying your git repo structure, pulling the api_check package one level down would be more appropriate. Simply create an api_check directory in the repo root and move all the python files from root in there. — hoefling
Yep, that ended up being how I fixed it. Moved it down a level. If you'd like to submit this as an answer, I'll accept it, or I can answer my own. Thanks! — klreeher

klreeher klreeher · Accepted Answer · 2018-10-15T14:07:11

Turns out, Azure expands the repo so that the highest level directory in the repo becomes the working directory, rather than being in the working directory.

To solve my problem, I moved my module to a second-level directory in my repo.

ex, GitHub repo layout:

$repo/api_check
$repo/api_check/auth
$repo/api_check/etc
$repo/api_check/__init__.py

becomes

$workingDirectory/api_check
$workingDirectory/api_check/auth
$workingDirectory/api_check/etc
$workingDirectory/api_check/__init__.py

There may be a way to make Azure expand the repo as a sub-directory in the working dir, but I haven't found it.

Azure DevOps can't find python module

3 Answers