Facing this error : container_linux.go:367: starting container process caused: exec: "python": executable file not found in $PATH: unknown

Question

So I'm a beginner in docker and containers and I've been getting this error for days now. I get this error when my lambda function runs a sagemaker processing job. My core python file resides in an s3 bucket. My docker image resides in ECR. But I dont understand why I dont get a similar error when I run the same processing job with a python docker image. PFB the python docker file that didnt throw any errors.

FROM python:latest
#installing dependencies
RUN pip3 install argparse
RUN pip3 install boto3
RUN pip3 install numpy
RUN pip3 install scipy
RUN pip3 install pandas
RUN pip3 install scikit-learn
RUN pip3 install matplotlib

I only get this error when i run this with a an ubunutu docker image with python3 installed. PFB the dockerfile which throws the error mentioned.

FROM ubuntu:20.04

RUN apt-get update -y
RUN apt-get install -y python3
RUN apt-get install -y python3-pip

RUN pip3 install argparse
RUN pip3 install boto3
RUN pip3 install numpy==1.19.1
RUN pip3 install scipy
RUN pip3 install pandas
RUN pip3 install scikit-learn

ENTRYPOINT [ "python3" ]

How do I fix this?

Abhinav Thakur Abhinav Thakur · Accepted Answer · 2021-05-28T20:53:32

Great question. Trust me, you've already solved 50% of the problem yourself by self-analysis.

You see, when you use an official Docker Image for Python, your Dockerfile is built on a pre-defined image; a Python Image in this case (or, just imagine that you're extending that base image with your own custom commands/layers). As it uses the existing Python image, it picks up all the pre-defined environment variables as well.

Whereas when you use an Ubuntu Docker Image, even though you're following all the right steps by installing dependencies, every layer of the Dockerfile runs in an intermediate container(imagine isolated independent shells), and is unable to set the $PATH to the Python bins.

The Solution :

Use a deterministic Python base image for your Dockerfile. You can do it by replacing latest in your FROM layer with a specific version of the Python Image. That's a better practice.

Yet, if you're keen on using an Ubuntu image, just add a Docker layer for ENV and mention the Python libs path. Your container will register it for future use once your own Docker image is ready.

You can refer to the official Docker documentation here. Search for ENV. If you have more questions, let me know.

Facing this error : container_linux.go:367: starting container process caused: exec: "python": executable file not found in $PATH: unknown

2 Answers