Python module import works for one file, fails for another

Question

I'm facing a very strange problem. I've got three files, the first contains the base class, from which the classes in the other two files inherit from.

The strange thing is, everything worked fine yesterday, but one of the files doesn't work anymore today. I haven't touched the imports in the meantime.

.
└── orangecontrib
    ├──__init__.py
    └── prototypes
        ├──__init__.py
        └── widgets
            ├──__init__.py
            ├── owpythagorastree.py
            ├── owclassificationpythagorastree.py
            └── owregressionpythagorastree.py

So the classification and regression classes need to inherit from the base class, and the imports are done in the exact same way:

owclassificationpythagorastree.py

...
from orangecontrib.prototypes.widgets.owpythagorastree import OWPythagorasTree
...

owregressionpythagorastree.py

...
from orangecontrib.prototypes.widgets.owpythagorastree import OWPythagorasTree
...

Yet when I try to run the two scripts from the command line (with python owregressionpythagorastree.py) the regression widget works fine, but the classification widget produces the following error:

Traceback (most recent call last): File "owclassificationpythagorastree.py", line 6, in from orangecontrib.prototypes.widgets.owpythagorastree import OWPythagorasTree ImportError: No module named 'orangecontrib.prototypes'

This has happened several times in other projects as well, but it eventually sorts itself out. But it bothers me that I don't know what is causing this.

I did try running this both from my regular machine and a python virtualenv, where I have the module installed (I did this with pip install -e . in the base directory).

I see no apparent reason for this behaviour and it is bothering me a lot, so any help or insight as to why this is happening and how to fix it would be appreciated.

EDIT

As requested, I ran import sys; print(sys.path) at the top of both scripts, and after running it through diff, they are both completely identical. Nevertheless, I am posting the results here.

['/home/pavlin/dev/orange3-prototypes/orangecontrib/prototypes/widgets', '/home/pavlin/dev/orange3', '/home/pavlin/dev/orange3env/lib/python3.5/site-packages/setuptools_git-1.1-py3.5.egg', '/home/pavlin/dev/orange-bio', '/home/pavlin/dev/orange3env/lib/python3.5/site-packages/pyqtgraph-0.9.10-py3.5.egg', '/home/pavlin/dev/orange3env/lib/python3.5/site-packages/requests-2.9.1-py3.5.egg', '/home/pavlin/dev/orange3env/lib/python3.5/site-packages/slumber-0.7.1-py3.5.egg', '/home/pavlin/dev/orange3env/lib/python3.5/site-packages/Genesis_PyAPI-1.2.0-py3.5.egg', '/usr/lib/python3.5/site-packages/qt_graph_helpers-0.1.3-py3.5-linux-x86_64.egg', '/home/pavlin/dev/orange3-prototypes', '/usr/lib/python3.5/site-packages', '/home/pavlin/dev/orange3env/lib/python35.zip', '/home/pavlin/dev/orange3env/lib/python3.5', '/home/pavlin/dev/orange3env/lib/python3.5/plat-linux', '/home/pavlin/dev/orange3env/lib/python3.5/lib-dynload', '/usr/lib64/python3.5', '/usr/lib/python3.5', '/usr/lib/python3.5/plat-linux', '/home/pavlin/dev/orange3env/lib/python3.5/site-packages', '/usr/lib/python3.5/site-packages/setuptools-18.7.1-py3.5.egg', '/home/pavlin/.local/lib/python3.5/site-packages']

There are __init__.py files in each directory. I'll add them to the question, to avoid confusion. This cannot be the issue, since the modules clearly work for one of the files, but not the other. — Pavlin
I can tell you this, for example if I try to import a module of mine on Desktop, if I try to import it on shell, I can't. Because shell working on another path, but if I import it to a .py file on Desktop, it works. Shell is on the Desktop too, So, try to check them are they actually running on that path. Staying on a path does not mean they are running on that path, that's the trick. — GLHF
What about from .owpythagorastree import OWPythagorasTree? Or even without the dot? — OneCricketeer
That results in a parent module '' not loaded error in both cases. — Pavlin

daphtdazz daphtdazz · Accepted Answer · 2016-06-05T12:55:53

It looks like your problem is a not-complete understanding of how python finds modules.

For an absolute import (i.e. one where you specify the name of the first module such as import mymodule, and don't use a period to do a relative import from the package to which the code belongs, such as from . import mymodule), the order of lookups is:

The current directory.
Any directories in PYTHONPATH in your environment.
Any installed system paths, which may be set in various ways.

To see your complete set of paths to import from, use:

import sys
print(sys.path)

Also, remember that a directory is only importable if it has an __init__.py file – you didn't say whether that was the case or not with your code.

Therefore, provided you have the following layout:

.
└── orangecontrib
    ├── __init__.py
    └── prototypes
        ├── __init__.py
        └── widgets
            ├── __init__.py
            ├── owpythagorastree.py
            ├── owclassificationpythagorastree.py
            └── owregressionpythagorastree.py

I would expect that if you run python orangecontrib/prototypes/widgets/owclassificationpythagorastree.py from the base directory ., both should work just fine, without needing to modify your PYTHONPATH, since python always looks in the current directory for absolutely imported modules first.

If you are running from the widgets directory, I would expect it not to work, unless you had first added the base directory to your PYTHONPATH.

As a general hint though, except for very small things, you should avoid mixing module code with script code, for exactly these reasons! Make separate python scripts that are designed to be run from a command line, and separate python module code that is designed to be imported. Avoid making modules that can also be run from the command line as scripts.

Python module import works for one file, fails for another

2 Answers