Azure devops pipeline - trigger only on another pipeline, NOT commit

6
votes

Requirement

So, there is some newish functionality in Azure DevOps which allows pipelines to trigger other pipelines and is documented here: https://docs.microsoft.com/en-us/azure/devops/pipelines/build/triggers?view=azure-devops&tabs=yaml#pipeline-triggers-1 It sounds great, apart from the fact I can't get the behaviour I need. I want 2 pipelines in the same repository:

  • Pipeline A: is only triggered by multiple other pipelines outside of its own repo, but in the same project. As a result of being triggered it makes a change to its own repo and therefore triggers pipeline B.
  • Pipleline B: is only triggered by changes to its own repo and when triggered goes ahead and does whatever it needs to do

Pipeline A Syntax

resources:
    pipelines:
    - pipeline: database
      source: database
      trigger:
        branches:
        - develop
        - release/*
        # The stages filter should work, according to: https://docs.microsoft.com/en-us/azure/devops/pipelines/build/triggers?view=azure-devops&tabs=yaml
        # However, this error occurs when specifying: /azure-pipelines.yml (Line: 8, Col: 15): Stage filters in pipeline resource database is not supported.
        #stages:
        #- Build
    - pipeline: auth
      source: auth
      trigger:
        branches:
        - develop
        - release/*
    - pipeline: api
      source: api
      trigger:
        branches:
        - develop
        - release/*
    - pipeline: web
      source: web
      trigger:
        branches:
        - develop
        - release/*
  ... multiple triggers - 9 in total
stages:
  ...

Current Behaviour

Pipeline A is not triggered by any of the other pipelines, but only on changes to its own repo. Since it makes changes to its own repo anyway, it triggers itself in an endless loop.

Questions / Comments

  • Is the syntax for Pipeline A correct?
  • From the documentation: "However, if the two pipelines use different repositories, then the triggered pipeline will use the latest version of the code from its default branch." I'm assuming this means the yaml pipeline from the default branch will be activated. And do we really have that little control? It'd be far better to specify the target branch in the pipeline declarations.
  • Is it possible to somehow get the source branch which triggered the pipeline?
  • Why don't stage filters work as documented?
  • In Pipeline A, to stop the looping, I tried using $(Build.TriggeredBy.DefinitionId) to check if that was the same as $(System.DefinitionId) and skip the build steps if so, but $(Build.TriggeredBy.DefinitionId) didn't have a value
  • I'm leaning towards getting the other pipelines to trigger Pipeline A if I can't get it to work.

Discovery

  • Adding trigger: none to the top of the pipeline A prevented it from running when commits were made to its repo, it just doesn't run at all currently!
  • In a simplified pipeline scenario in a separate account, I managed to get triggered builds working, with 2 pipelines in the same repo and found:
    • The yaml pipeline file executed is on the same branch as the commit on the triggering pipeline
    • The code checked out is also from the same branch as the commit on the triggering pipeline
    • Manual pipeline execution from the GUI does not trigger dependent pipelines
    • The dependent pipeline is triggered and queued immediately the first starts
    • I couldn't get branch exclusion to work: the pipeline triggered regardless of the exclusion clause
  • Running a simplified pipeline scenario in a separate account, with pipeline A in repo C and dependant pipeline B in repo D (same project), I am unable to get pipeline A to trigger pipeline B so far (my original scenario)
  • Oh great joy :-) there's an az azure devops command line extension which has got pipeline support and lets you trigger a pipeline:
3
azure-devopsazure-pipelinesbuild-triggers
6 questions in one, whoa. I dont see anything wrong with your syntax. wrong pipeline names? no idea – 4c74356b41
@vipes thanks for sharing. I am struggling similar thing. What did you do to: - run pipeline A only once when other pipeline has finished? – wolszakp
Sorry @wolszakp - I didn't get notification of your comment. As per my answer below, I created a reusable yaml template which uses the Azure CLI to trigger pipeline A and is called by a number of other pipelines at the end of their processing. Once pipeline A has been triggered, it updates the repo it's contained in, triggering pipeline B because of the commit. – vipes

3 Answers

9
votes

Working Solution

Because all of my builds are centralised in one pipeline template, I changed this template to trigger my pipeline A on successful publishing of an artifact. Here's the pipeline trigger code which is pretty much verbatim from (https://docs.microsoft.com/en-us/azure/devops/cli/azure-devops-cli-in-yaml?view=azure-devops), apart from the last couple of steps:

# Updating the python version available on the linux agent
    - task: UsePythonVersion@0
      displayName: Upgrade build agent Python version
      inputs:
        versionSpec: '3.x'
        architecture: 'x64'

    # Updating pip to latest
    - script: python -m pip install --upgrade pip
      displayName: 'Upgrade pip'

    # Updating to latest Azure CLI version.
    - script: pip install --pre azure-cli --extra-index-url https://azurecliprod.blob.core.windows.net/edge
      displayName: 'Upgrade azure cli'

    - script: az --version
      displayName: 'Show Azure CLI version'

    - script: az extension add -n azure-devops
      displayName: 'Install Azure DevOps Extension'

    - script: echo ${AZURE_DEVOPS_CLI_PAT} | az devops login
      env:
        AZURE_DEVOPS_CLI_PAT: $(System.AccessToken)
      displayName: 'Login Azure DevOps Extension'

    - script: az devops configure --defaults organization=$(System.TeamFoundationCollectionUri) project="$(System.TeamProject)" --use-git-aliases true
      displayName: 'Set default Azure DevOps organization and project'

    - script: |
        set -euo pipefail
        if [[ "$(Build.SourceBranch)" == *"/release/"* ]]; then
          branchName="master"
        else
          branchName="develop"
        fi
        commandLine="--branch $branchName --name <YourPipelineName>"
        echo "Triggering release creation with: az pipelines run $commandLine"
        az pipelines run $commandLine
      displayName: Trigger release build for internal (develop) and external (release) builds

Caveats

  • Change <YourPipelineName> as appropriate, and your branch name handling is going to be different to mine
  • The Azure CLI upgrade takes 1.5 minutes, so just delete the first 3 steps if you want to significantly speed it up
  • I'd rather the triggered pipeline declared its own triggers, as it's easier to maintain if you can see what's causing it to trigger, but hey-ho
2
votes

I was running into the same issue. I found solution over here: https://developercommunity.visualstudio.com/content/problem/864701/pipeline-trigger-not-working-as-expressed-in-docum.html?childToView=897210#comment-897210

In shortcut, when I've changed defaultBranch in my dependent pipeline to my working branch triggering start working.: You need to go:

Edit -> Triggers (3 dots in right upper corner) -> YAML -> Get sources -> Default branch for manual and scheduled builds

Over there put a branch in which your yaml pipeline exists. Default branch

0
votes

If you're not publishing an artifact from the triggering pipeline, it won't trigger the triggered pipeline. I have created a minimum viable product where this is working, and I have explained the process in this answer.