Module Not Found when using pytest

Question

I have a virtualenv that uses Python 3 and am currently using pytest for testing. I've packaged a program called "process_expiring_passwords" and import this package from my general script file called "pep.py". I've also used pip in my virtualenv to install my process_expiring_passwords package.

When I go to run my tests, however, I get the following error:

ImportError while importing test module '/home/username/projects/process-expiring-passwords/tests/test_process_expiring_passwords.py'.

Hint: make sure your test modules/packages have valid Python names.

Traceback: tests/test_process_expiring_passwords.py:3: in from process_expiring_passwords import ProcessExpiringPasswords process_expiring_passwords/process_expiring_passwords.py:16: in from process_expiring_passwords.models.ProcessExpiringPasswordsLog

\ E ModuleNotFoundError: No module named 'process_expiring_passwords.models'; 'process_expiring_passwords' is not a package

Is there a particular reason why I can use my package from my pep.py general script file while pytest is unable to find said package?

The essential parts of the code related to this issue are as follows:

pep.py:

from process_expiring_passwords.process_expiring_passwords \
    import ProcessExpiringPasswords

pep = ProcessExpiringPasswords()
pep.process()

process_expiring_passwords/process_expiring_passwords.py:

from process_expiring_passwords.models.ProcessExpiringPasswordsLog \
    import ProcessExpiringPasswordsLog

class ProcessExpiringPasswords:
    def __init__(self):
        self.example = 0
    def process():
        pepl = ProcessExpiringPasswordsLog()

process_expiring_passwords/models/ProcessExpiringPasswordsLog.py:

class ProcessExpiringPasswordsLog():
    def __init__(self):
        self.example = 0

process_expiring_passwords/setup.py:

# ref: https://github.com/pypa/sampleproject/blob/master/setup.py
# ref: http://python-packaging.readthedocs.io/en/latest/minimal.html
from setuptools import setup

setup(
    name='process_expiring_passwords',
    version='0.0.1',
    packages=find_packages(),
    description='Process expiring passwords',
    license='MIT',
    install_requires=[
        'apiclient',
        'google-api-python-client',
        'httplib2',
        'mysqlclient',
        'oauth2client',
        'requests',
        'SQLAlchemy',
    ],
)

tests/test_process_expiring_passwords.py:

from process_expiring_passwords import ProcessExpiringPasswords

def test_pep():
    pep = ProcessExpiringPasswords()
    assert pep is not None

Project structure:

process_expiring_passwords
    pep.py
    process_expiring_passwords
        __init__.py
        models
            __init__.py
            ProcessExpiringPasswordsLog.py
        process_expiring_passwords.py
        setup.py
    tests
        test_process_expiring_passwords.py

Notes:

Updated code example and setup.py based on feedback from Matt Messersmith.

The only reason that pytest doesn' t find your module ist that it's not in your PATH. As you said you installed it (I suppose with pip -e . the only reason I can think of is that pytest is not installed in your virtualenv and thus you run the "main python" variant which can not find your module in the PATH. Install pytest in your virtualenv and it should work. — MrLeeh
@MrLeeh Thank you for the comment/suggestion! I've double-checked, and the pytest that I'm currently using already appears to be in my virtualenv. — summea
@hoefling Thanks for the suggestions! I've tried the conftest.py approach just now but I get the same results that I received when I tried the last two of your suggestions previously. I still end up getting a similar (but shortened) error: Traceback: tests/test_process_expiring_passwords.py:3: in <module> from process_expiring_passwords import ProcessExpiringPasswords E ImportError: cannot import name 'ProcessExpiringPasswords' — summea
disregard my previous comment, I now see what the issue is. You have the package having the same name as the module, so when you do import process_expiring_passwords inside process_expiring_passwords/process_expiring_passwords.py, you don't import the parent package. Add lines import process_expiring_passwords; print(process_expiring_passwords.__file__) to process_expiring_passwords.py and verify that you import the module instead. — hoefling
Overall, the package structure looks a bit messy to me, can you enhance the code in question to a minimal reproducible example? Your setup script resides in the package process_expiring_passwords which it presumably should collect and install, how do you pass the package to packages list? Do you use setuptools.find_packages for that? The same naming of the package and module may be only a complication that distracts from the actual error. — hoefling

Matt Messersmith Matt Messersmith · Accepted Answer · 2018-10-26T16:57:38

Your code doesn't quite work as posted, I modified it to have the self keyword in your constructors and member functions. After that, I could get pep.py to run.

Is there a particular reason why I can use my package from my pep.py general script file while pytest is unable to find said package?

Yes, when you are in your top level process_expiring_passwords directory, you can run pep.py, because you'll be getting a "local" import of the package.

The fact that the test doens't run has little to do with pytest, and more to do with what your import statement looks like and the fact that you're not building a package:

from process_expiring_passwords import ProcessExpiringPasswords

You haven't created a package called process_expiring_passwords. You can verify this by doing python setup.py bdist_wheel (or just do sdist and unzip the tarball), unzipping the wheel, and noting that there are no packages present.

Typically, you need to install the packages python setup.py install (this plops the distributable in your site-packages), and you also have to name the packages in setup.py (find_packages() is your friend), and then your test should work (more or less) as expected.

HTH.

Module Not Found when using pytest

2 Answers