Installing packages using tox from private git repo on Azure DevOps build pipeline using $(System.AccessToken)

2
votes

During a build on Azure DevOps, I'm trying to install a package from a private git repo using tox, but it errors out with

Collecting git+https://****@<company>.visualstudio.com/<team>/_git/<repo_name>@<branch>
  Cloning https://****@<company>.visualstudio.com/<team>/_git/<repo_name> (to revision <branch>) to c:\users\vssadm~1\appdata\local\temp\pip-req-build-xpl77xit
  Running command git clone -q 'https://****@<company>.visualstudio.com/<team>/_git/<repo_name>' 'C:\Users\VSSADM~1\AppData\Local\Temp\pip-req-build-xpl77xit'
  fatal: Cannot prompt because user interactivity has been disabled.
  fatal: could not read Password for 'https://$(System.AccessToken)@<company>.visualstudio.com': terminal prompts disabled

The tox.ini is setup with

passenv = *
deps = 
  git+https://$(System.AccessToken)@<company>.visualstudio.com/<team>/_git/<repo_name>@<branch>

I've ensured that the Azure DevOps build agent user has read permissions on the <repo_name> repository and that the "Allow script to access the OAuth token" is checked.

When I create a script task to simply run pip install git+https://$(System.AccessToken)@<company>.visualstudio.com/<team>/_git/<repo_name>@<branch> it successfully installs.

Is there a way for to use $(System.AccessToken) with tox?

1
pythonazure-devopstox
tox.readthedocs.io/en/latest/… – phd
Thanks @phd. I got hung up on Azure DevOp's special variable notation. Looking at the output, it looks like the build is substituting $(System.AccessToken), but it's just substituting in the tox output, not what tox is running. This is confirmed in the last fatal output message. – Adam

1 Answers

2
votes

In tox.ini, instead of using $(System.AccessToken) use {env:SYSTEM_ACCESSTOKEN}.

Azure DevOps pipelines allows for getting the access token either from their predefined pipeline variable System.AccessToken or an environment variable. The pipeline variable can be used in the build definition or in a task.

In the case of attempting to use $(System.AccessToken) in tox.ini, that build variable isn't being swapped since tox is reading the file directly and just using that string without any substitution. It may seem like it is substituting since the logs show Collecting git+https://****@<company>.visualstudio.com, however this is just DevOps seeing Collecting git+https://$(System.AccessToken)@<company>.visualstudio.com and doing the swap before it prints to the log. Notice too in the last fatal log it writes $(System.AccessToken) fatal: could not read Password for 'https://$(System.AccessToken)@<company>.visualstudio.com': terminal prompts disabled.

Azure DevOps makes these pipeline variables available as an environment variable. For System.AccessToken, this environment variable is SYSTEM_ACCESSTOKEN. To access environment variables in tox, the syntax is {env:KEY}. So to access System.AccessToken via tox, use {env:SYSTEM_ACCESSTOKEN}:

passenv = *
deps = 
  git+https://{env:SYSTEM_ACCESSTOKEN}@<company>.visualstudio.com/<team>/_git/<repo_name>@<branch>