Dockerfile Build Fails - Pipenv and Pyenv Multiple Versions of Python Found

Question

Question: How can I fix the Dockerfile to properly freeze requirements.txt and successfully build?

I am working through deploying a Dockerfile of a Python script utilizing Pyenv and Pipenv for local development.

On the build step where the Piplock file is frozen to requirements.txt, I receive the following error:

Error: Invalid value for "--python": Expected Python at path /Users/jz/.local/share/virtualenvs/quanter-TP0oWHoL/bin/python3 does not exist

My Dockerfile is:

FROM python:3.7
RUN pip install pipenv
COPY Pipfile* /tmp/
RUN cd /tmp && pipenv --python /Users/x/.local/share/virtualenvs/quanter-TP0oWHoL/bin/python3 lock --requirements > requirements.txt
ENV RDKAFKA_INSTALL=system
RUN pip install -r /tmp/requirements.txt
COPY . /tmp/app/
RUN pip install /tmp/app/
CMD ["python", "./tmp/app/main.py"]

Creation of the local Pipenv environment provided this information about the interpreter, (which was used in the Dockerfile):

Using /usr/local/opt/pyenv/versions/3.8.0/bin/python3 (3.8.0) to create virtualenv… ⠙ Creating virtual environment...Using base prefix '/usr/local/opt/pyenv/versions/3.8.0' New python executable in /Users/x/.local/share/virtualenvs/quanter-TP0oWHoL/bin/python3 Also creating executable in /Users/x/.local/share/virtualenvs/quanter-TP0oWHoL/bin/python

Pyenv is using 3.8.0 locally:

pyenv versions
system
3.7.5
* 3.8.0 (set by /Users/x/Projects/quanter/.python-version)

Any help getting this working would be greatly appreciated! Thank you.

Why do you need manually creating lock: converting requirement from pipenv and making a lock of it? Pipenv does this for you, isn't? — funnydman
@funnydman While I want to use Pipenv for its virtual environment in local development, I don’t want a virtual environment inside of the Docker environment, thus confusing the purpose of Docker being the environment. I just want to use Pip in the Docker container and install the dependencies systemically from a requirements file. no virtual environment needed. — Liquidgenius
@funnydman as it stands right now Pipenv doesn’t work anyways due to the issue I’ve outlined with not being able to locate the Pyenv interpreter. — Liquidgenius

funnydman funnydman · Accepted Answer · 2019-12-02T07:24:47

The original error came from the idea that pipenv can not find python executable on the given path. It is hardcoded, but on every fully new build, pipenv will create env with a different name. Take a look at this interesting discussion: How does pipenv know the virtualenv for the current project?, it seems like it could be predictable, but there should be a better solution.

Firstly, you do not need to specify a path to python executable directly because python image comes only with one system python installation, e.g. it will be available by default via python (of course you can create different environments with different versions of python by hand).

Secondly, you can handle pipenv in docker env in a more nicer and preferable way, instead of converting pipenv into pip like flow. Take a look at the example below:

FROM python:3.7

COPY Pipfile /
COPY Pipfile.lock /

RUN pip3 install pipenv \
    && pipenv install --system --deploy --ignore-pipfile

Several notes worthing to consider:

Commit Pipfile.lock into VCS. In this case, you can be sure that in every environment you have exactly the same versions of packages and its dependencies.
You may be interested in --system option. That says to install dependencies into a system instead of virtualenv (this is what you expect). But according to this answer: https://stackoverflow.com/a/55610857/9926721 It is not officially recommended. But still, a quite popular project wemake-django-template uses it as well.
Consider splitting up environments. There is useful flag --dev for it. No more several requirement files. Handle it more efficient way.

Dockerfile Build Fails - Pipenv and Pyenv Multiple Versions of Python Found

1 Answers