python - How to state in requirements.txt a direct github source

548
votes

I've installed a library using the command

pip install git+git://github.com/mozilla/elasticutils.git

which installs it directly from a Github repository. This works fine and I want to have that dependency in my requirements.txt. I've looked at other tickets like this but that didn't solve my problem. If I put something like

-f git+git://github.com/mozilla/elasticutils.git
elasticutils==0.7.dev

in the requirements.txt file, a pip install -r requirements.txt results in the following output:

Downloading/unpacking elasticutils==0.7.dev (from -r requirements.txt (line 20))
  Could not find a version that satisfies the requirement elasticutils==0.7.dev (from -r requirements.txt (line 20)) (from versions: )
No distributions matching the version for elasticutils==0.7.dev (from -r requirements.txt (line 20))

The documentation of the requirements file does not mention links using the git+git protocol specifier, so maybe this is just not supported.

Does anybody have a solution for my problem?

9
pythongithubpiprequirements.txt

9 Answers

592
votes

Normally your requirements.txt file would look something like this:

package-one==1.9.4
package-two==3.7.1
package-three==1.0.1
...

To specify a Github repo, you do not need the package-name== convention.

The examples below update package-two using a GitHub repo. The text between @ and # denotes the specifics of the package.

Specify commit hash (41b95ec in the context of updated requirements.txt):

package-one==1.9.4
git+git://github.com/path/to/package-two@41b95ec#egg=package-two
package-three==1.0.1

Specify branch name (master):

git+git://github.com/path/to/package-two@master#egg=package-two

Specify tag (0.1):

git+git://github.com/path/to/[email protected]#egg=package-two

Specify release (3.7.1):

git+git://github.com/path/to/package-two@releases/tag/v3.7.1#egg=package-two

Note that #egg=package-two is not a comment here, it is to explicitly state the package name

This blog post has some more discussion on the topic.

371
votes

“Editable” packages syntax can be used in requirements.txt to import packages from a variety of VCS (git, hg, bzr, svn):

-e git://github.com/mozilla/elasticutils.git#egg=elasticutils

Also, it is possible to point to particular commit:

-e git://github.com/mozilla/elasticutils.git@000b14389171a9f0d7d713466b32bc649b0bed8e#egg=elasticutils
193
votes

requirements.txt allows the following ways of specifying a dependency on a package in a git repository as of pip 7.0:1

[-e] git+git://git.myproject.org/SomeProject#egg=SomeProject
[-e] git+https://git.myproject.org/SomeProject#egg=SomeProject
[-e] git+ssh://git.myproject.org/SomeProject#egg=SomeProject
-e [email protected]:SomeProject#egg=SomeProject (deprecated as of Jan 2020)

For Github that means you can do (notice the omitted -e):

git+git://github.com/mozilla/elasticutils.git#egg=elasticutils

Why the extra answer?
I got somewhat confused by the -e flag in the other answers so here's my clarification:

The -e or --editable flag means that the package is installed in <venv path>/src/SomeProject and thus not in the deeply buried <venv path>/lib/pythonX.X/site-packages/SomeProject it would otherwise be placed in.2

Documentation

90
votes

First, install with git+git or git+https, in any way you know. Example of installing kronok's branch of the brabeion project:

pip install -e git+https://github.com/kronok/brabeion.git@12efe6aa06b85ae5ff725d3033e38f624e0a616f#egg=brabeion

Second, use pip freeze > requirements.txt to get the right thing in your requirements.txt. In this case, you will get

-e git+https://github.com/kronok/brabeion.git@12efe6aa06b85ae5ff725d3033e38f624e0a616f#egg=brabeion-master

Third, test the result:

pip uninstall brabeion
pip install -r requirements.txt
19
votes

Since pip v1.5, (released Jan 1 2014: CHANGELOG, PR) you may also specify a subdirectory of a git repo to contain your module. The syntax looks like this:

pip install -e git+https://git.repo/some_repo.git#egg=my_subdir_pkg&subdirectory=my_subdir_pkg # install a python package from a repo subdirectory

Note: As a pip module author, ideally you'd probably want to publish your module in it's own top-level repo if you can. Yet this feature is helpful for some pre-existing repos that contain python modules in subdirectories. You might be forced to install them this way if they are not published to pypi too.

7
votes

None of these answers worked for me. The only thing that worked was:

git+https://github.com/path_to_my_project.git

No "e", no double "git" and no previous installs necessary.

5
votes

Github has a zip endpoint that in my opinion is preferable to using the git protocol. The advantages are:

  • You don't have to specify #egg=<project name>
  • Git doesn't need to be installed in your environment, which is nice for containerized environments
  • It works much better with pip hashing and caching
  • The URL structure is easier to remember and more discoverable

You usually want requirements.txt entries to look like this, e.g. without the -e prefix:

https://github.com/org/package/archive/1a58aa586efd4bca37f2cfb9d9348958986aab6c.zip

To install from main branch:

https://github.com/org/package/archive/main.zip
3
votes

I'm finding that it's kind of tricky to get pip3 (v9.0.1, as installed by Ubuntu 18.04's package manager) to actually install the thing I tell it to install. I'm posting this answer to save anyone's time who runs into this problem.

Putting this into a requirements.txt file failed:

git+git://github.com/myname/myrepo.git@my-branch#egg=eggname

By "failed" I mean that while it downloaded the code from Git, it ended up installing the original version of the code, as found on PyPi, instead of the code in the repo on that branch.

However, installing the commmit instead of the branch name works:

git+git://github.com/myname/myrepo.git@d27d07c9e862feb939e56d0df19d5733ea7b4f4d#egg=eggname
3
votes

It seems like this is also a valid format:

gym-tictactoe @ git+https://github.com/haje01/gym-tictactoe.git@84e22fc28fe192ba0040bdd56a697f63d3d4a3d5

If you do a pip install "git+https://github.com/haje01/gym-tictactoe.git", then look at what got installed by running pip freeze, you will see the package described in this format and can copy and paste into requirements.txt.