ModuleNotFoundError in Jupyter with pipenv

Question

I am at a Python boot camp this weekend but I have not been able to even use Python on my computer because of this issue. All my instructors are stumped too.

The issue is that I get the ModuleNotFoundError on Jupyter with multiple different packages, including Pandas and Requests (but oddly enough, BeautifulSoup and CSV work fine.)

Here is how I start a new Jupyter file:

• Create a new directory
• Install jupyter and pandas with this command: pipenv install jupyter pandas
• Activate virtual environment: pipenv shell
• Launch Jupyter: jupyter notebook
• Create new Python 3 notebook
• At this point, I try a command like import pandas as pd and get back the ModuleNotFoundError.

I am using Python version 3.6.5.

Attempts to fix this that have failed:

• double-checked that pandas is installed in my virtual environment with pip graph

• created completely new directory pipenv install jupyter pandas --skip-lock

• Uninstalled everything system-wide with these commands:

pip freeze > requirements.txt
pip uninstall -r requirements.txt -y

• Updated pandas
• Used virtualenv instead of pipenv

virtualenv first-python-notebook
cd first-python-notebook
cd Scripts
activate
cd ..
pip install jupyter pandas

I tested that pandas could be imported when I used python in the command shell (yes) -- still didn't work on Jupyter.

My instructor thinks the issue is that system-wide packages are interfering with virtual ones but we have been working for hours and cannot figure out how to fix this.

Any help would be greatly appreciated. Please include detailed instructions as I am a beginner.

Answer 1

If you're getting 'ModuleNotFoundError: No module named xxyyzz' in jupyter, but the module can be imported by running python via the pipenv shell (pipenv run python -c "import xxyyzz; print(xxyyzz.__version__)":

• it's probably jupyter's python path isn't set properly in the kernel config file: ..\jupyter\kernels\<myProjectName>\kernel.json
• the kernel needs to be created within the pipenv shell to pick up the right path

With a fresh pipenv install:

1. pip install pipenv
2. cd <project directory>
3. export PIPENV_VENV_IN_PROJECT=1 # creates .venv in project directory
4. pipenv --python=/path/to/python --site-packages # use python executable for your system or environment
5. pipenv shell # work in project's virtual environment
6. python -m ipykernel install --user --name=<myProjectName> # create jupyter kernel for project
7. exit # exit project's virtual environment
8. pipenv run jupyter notebook # start jupyter from project directory
9. in jupyter, choose the kernel "myProjectName"

this post provides additional explanations

Answer 2

Why don't you try to install ipykernel with Anaconda virtual env? It'll will be more easy to handle.

If you haven't previously used Anaconda before, just go to the official website https://www.anaconda.com/download/ and download the newest version for your OS. Then, follow these steps.

1. Execute Anaconda prompt.
2. Type 'conda create -y -n $ENVIRONMENT_NAME ipykernel'
3. Type 'conda activate $ENVIRONMENT_NAME'
4. Type 'conda install -y $PACKAGES_TO_BE_INSTALLED'
5. Type 'python -m ipykernel install --user --name $NAME --display-name $IPYKERNEL_NAME'

This ipykernel name will be presented on your list of kernels in jupyter notebook. You can findout the list of kernels installed by typing jupyter kernelspec list.

Hope this helps!

Answer 3

Thanks for the advice. However, I was advised specifically not to install Anaconda -- can't quite remember the reason but I think it's because, basically, if I ever decided I wanted to use something else then it would be a real headache to switch. I'm happy to hear your reasoning if you disagree with that.

I ended up solving the issue by uninstalling every package both within the virtual environment and the larger computer system, then re-installing it in both places. It worked, but I'm sort of confused as to what the point of a virtual environment is, if I still had to install everything twice.