pip: prioritize a private pypi server

Question

I'm using Artifactory to host a large number of python modules. A couple of times, a new python module on https://pypi.python.org/pypi will be uploaded with the same name. This causes problems when someone tries to install one of my modules from Artifactory using pip. By default, pip looks for public packages first.

Here's what I've tried:

1. I've tried modifying my .pypirc file accordingly,

[distutils]
index-servers=
        artifactory
        pypi

with appropriate entries for the two index servers, but I'm fairly certain that pip is ignoring this information.

2. I've tried manually specifying which index server to use.

I can specify --index-url or --extra-index-url but the latter does not help when the public package has a higher version number than my private package. If I specify the former, then public dependencies can't be found! It's like I can't win.

3. I've tried specifying dependency_links in setup.py.

... but this is deprecated as per this answer: pip ignores dependency_links in setup.py

How do I configure pip to prefer my Artifactory repository over the public one?

Oleh Prypin Oleh Prypin · Accepted Answer · 2021-05-07T22:47:18

This cannot be achieved.

pip devs have stated that all package indexes always have equal priority, and that this will not change.

https://github.com/pypa/pip/issues/8606#issuecomment-788754817
https://github.com/pypa/pip/issues/8606#issuecomment-665554122

The choice of index to download from is made per-package.

Among two indexes (regardless how they're specified), if one has a greater version of the package, that index will be chosen. And if the versions are equal, the choice is done in some unspecified and unstable order that might happen to work out for you by accident.

If you are in that situation where things seem to "work", note that the other index (if not controlled by you) can publish a higher version at any time and users will be getting that one instead, implicitly.
The same applies when only one of the two indexes currently have some package: whenever the other index decides to publish a package under that name, users can be implicitly switched to that index.

https://github.com/pypa/pip/issues/8606#issuecomment-776623044

The solution recommended by pip devs is to directly use your own package index for everything. Normally that might mean mirroring all of PyPI first, but actually that's not necessary, because there are implementations (e.g. "simpleindex") that can proxy requests to a different destination explicitly based on which package it is. So that would be the actual complete solution.

https://github.com/pypa/pip/issues/8606#issuecomment-835201448

Another common suggestion is to register in advance the same package name on PyPI and just not upload anything there, just so you own it, thus preventing the name-squatting attack. (Though that only accounts for the case where PyPI is the only public index involved. Generally, you'd need to predict all public indexes you'll ever rely on and register on all of them).

https://github.com/pypa/pip/issues/8606#issuecomment-822701285

Disclaimer: I am not involved with Pip and only summarizing the information I've read in the linked issue and related ones.

pip: prioritize a private pypi server

2 Answers