Pip authenticate task creates new value for PIP_INDEX_URL/PIP_EXTRA_INDEX_URL each time the pipeline runs, preventing docker to reuse cached layer

Question

I am using azure devops classic build pipeline's pip authenticate task to create a connection to a private package feed. This task creates environment variables PIP_INDEX_URL and PIP_EXTRA_INDEX_URL.

I am passing the above value as an argument into the dockerfile to configure the package source credentials.

ARG EXTRA_INDEX_URL
ENV ENV_EXTRA_INDEX_URL ${EXTRA_INDEX_URL}
RUN pip config set global.extra-index-url "$ENV:ENV_EXTRA_INDEX_URL"
RUN pip install -r requirements.txt

Azure devops Pip authenticate task creates new value for PIP_INDEX_URL/PIP_EXTRA_INDEX_URL each time the pipeline runs preventing docker to reuse cached layer. Is there any way to allow docker image layer re-use when deriving the credentials using the pip authenticate task?

Answer 1

Azure devops Pip authenticate task creates new value for PIP_INDEX_URL/PIP_EXTRA_INDEX_URL each time the pipeline runs preventing docker to reuse cached layer. Is there any way to allow docker image layer re-use when deriving the credentials using the pip authenticate task?

Sorry but I'm afraid the answer is No. It's expected behavior about docker that cached layer won't be reused when the ARG's value gets changed every time. Since the pip authenticate task always creates new temporary credentials each time when the pipeline is triggered, we can't avoid creating the new layers.

The suggestion is that you can put the script above in Dockerfile just before where you need to use those python packages, instead of the very top. This can save some time when the commands above your script can reuse cached layers. You can check this similar post.