How can I install packages hosted in a private PyPI using setup.py?

30
votes

I'm trying to write the setup.py install file for a private project, which has both public and private dependencies. The public ones are hosted on PyPI, whereas the private ones are hosted on a server running simplepypi.

I would like both public and private dependencies to be resolved and fetched during installation.

I thus added my dependencies to setup.py:

setup(
    ...
    install_requires = [
        # public dependencies
        'argparse==1.2.1',
        'beautifulsoup4==4.1.3',
        'lxml==3.1.0',
        'mongoengine==0.8.2',
        'pymongo==2.5.2',
        'requests==1.1.0',
        'Cython==0.18',
        # private dependencies
        'myprivatepackage1',
        'myprivatepackage2'
    ],
    dependency_links=['http://pypi.myserver.com/packages'],
    ...
)

I build the package tarball using the command python setup.py sdist and install it in an activated virtualenv using pip install --verbose path/to/tarball.tar.gz.

However, the pip log lines do not mention my private PyPI server anywhere, and https://pypi.python.org/simple/ seems to have been queried twice.

Running setup.py egg_info for package from file:///home/b/code/mapado/mypackage/dist/mypackage-0.5.1.tar.gz
    running egg_info
    creating pip-egg-info/mypackage.egg-info
    writing requirements to pip-egg-info/mypackage.egg-info/requires.txt
    writing pip-egg-info/mypackage.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/mypackage.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/mypackage.egg-info/dependency_links.txt
    writing manifest file 'pip-egg-info/mypackage.egg-info/SOURCES.txt'
    warning: manifest_maker: standard file '-c' not found

    reading manifest file 'pip-egg-info/mypackage.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'pip-egg-info/mypackage.egg-info/SOURCES.txt'
Downloading/unpacking myprivatepackage (from mypackage==0.5.1)
  Could not fetch URL https://pypi.python.org/simple/myprivatepackage/: HTTP Error 404: Not Found (myprivatepackage does not have any releases)
  Will skip URL https://pypi.python.org/simple/myprivatepackage/ when looking for download links for myprivatepackage (from mypackage==0.5.1)
  Could not fetch URL https://pypi.python.org/simple/myprivatepackage/: HTTP Error 404: Not Found (myprivatepackage does not have any releases)
  Will skip URL https://pypi.python.org/simple/myprivatepackage/ when looking for download links for myprivatepackage (from mypackage==0.5.1)
  Could not find any downloads that satisfy the requirement myprivatepackage (from mypackage==0.5.1)
Cleaning up...

What am I missing?

Thank you very much!

3
pythonsetuptoolspypi

3 Answers

20
votes

it looks like you didnt specify your host like the doc of simplepy said you need to setup your ~/.pypirc with the good hostname like

To use it run "simplepypi". You can upload packages by:

Modify your ~/.pypirc so it looks like:

    [distutils]
    index-servers =
        pypi
        local

    [local]
    username: <whatever>
    password: <doesn't matter, see above>
    repository: http://127.0.0.1:8000

    [pypi]
    ...

then you'll upload your package on it 

python setup.py sdist upload -r local

and could install it from there 

pip install -i http://127.0.0.1:8000/pypi <your favorite package>

Hope this will help.

0
votes

dependency_links is ignored by default (at least in pip 9.0.1)

In order for it to reach out to your sever you need to add --process-dependency-links

I believe pip 10 will bring a new mechanism, but for now this has got it working for me

I also had to update dependency_links to include the package name, for example:

dependency_links=[
    "http://internal-pyp:5678/simple/your_package_name"
]
0
votes

You could make your package as a normal pip package and publish it to the private repo. To install it, you can specify global option --extra-index-url in the config file:

$ cat ~/.pip/pip.conf
[global]
extra-index-url = https://...