python pip - development mode with local dependencies

6
votes

Background

I have two python projects. Project A and Project B.

Each project has its own virtual environment and its own setup.py.

Both projects are not pure py files and has "build" process like building extensions, generate source etc.

A is dependent on B (setup.py install_requires points on B).

Each project is built and published/distributed as a wheel to pypi on-premise repository.

pip install w/wo -e (development mode) of project A, nicely installs project B into project A venv site-packages.

So far everything is working nicely and "by the book".

Now, my story get complicated ...

I would like to develop the two projects together without having to publish B in order A to consume it.

As example, I would like to:

  1. Change something in B.
  2. build B (setup.py build).
  3. NOT publish B as wheel to pypi.
  4. Goal - Project A will "know" the modified Project B. WITHOUT manually tweaking sys.path.

I actually want the pip install -e of project A to install project B in development mode too.

After some reading I understood (hopefully correctly) that one can define a "distribution" as a local source folder by --find-links flag.

I defined project B root folder in --find-links.

I tried it with (on project A setup.py folder):

  1. pip install . -e --find-links = file:///path/to/B

  2. pip install . -e --find-links = git+file://path/to/B

Both did not work.

BTW, Puting in the path to B wheel or egg of B,

ex: pip install . -e --find-links = file:///path/to/B.whl

did work but this is not what I am looking for.

Hope you are still with me :-) (sorry for the tedious story)

What am I missing?

Tx

1
pythonpython-3.xpipsetuptoolspython-wheel
this question might help you... try this pip install git+https://github.com/user/project.git@version or, if slow, try installing from zip You can also have different files for development and production dependencies, installing them recursively - leoschet
--find-links makes pip to look into either a directory with tars/eggs/wheels or an HTML file with links to packages in downloadable forms. You cannot just point it to a local project. - phd
Leoschet - this does not answer my usecase. My dependent is local folder and not remote or zip. - Lior Cohen
@Phd - so no other way to resolve it but writing "out of pip" scripts to pip -e the dependencies too or to add b to sys.path? - Lior Cohen
@hoefling - Thanks. I wish there was a way. BTW, I did not managed to declare dependency to the local source files at all. I tried to point find-links to the source setup.py folder and it just complained it can not find a distribution there. All the examples I see online are git+http. Do you know a working example that does this? - Lior Cohen

1 Answers

0
votes

Let me try and restate the problem:

  • You work on two python packages, package A and package B. A depends on B.
  • You have chosen to develop these two packages in separate virtual environments.
  • You would like to make some local changes to package B and have package A use the modified version of package B.

I apologize if I'm missing something here, but why not simply install both packages in the same virtual environment, which will make this problem go away?

I.e. after creating your environment, you install package B in editable mode, followed by installing package A. Any changes in B will be picked up by A, no changes needed.