75
votes

When making changes to YAML-defined Azure DevOps Pipelines, it can be quite tedious to push changes to a branch just to see the build fail with a parsing error (valid YAML, but invalid pipeline definition) and then try to trial-and-error fix the problem.

It would be nice if the feedback loop could be made shorter, by analyzing and validating the pipeline definition locally; basically a linter with knowledge about the various resources etc that can be defined in an Azure pipline. However, I haven't been able to find any tool that does this.

Is there such a tool somewhere?

5

5 Answers

27
votes

UPDATE: This functionality was removed in Issue #2479 in Oct, 2019


You can run the Azure DevOps agent locally with its YAML testing feature.

  1. From the microsoft/azure-pipelines-agent project, to install an agent on your local machine.
  2. Then use the docs page on Run local (internal only) to access the feature that is available within the agent.

This should get you very close to the type of feedback you would expect.

21
votes

FYI - this feature has been removed in Issue #2479 - remove references to "local run" feature

Hopefully they'll bring it back later considering Github Actions has the ability to run actions locally

4
votes

Azure DevOps has provided a run preview api endpoint that takes a yaml override and returns the expanded yaml. I added this support to the AzurePipelinePS powershell module. The command below will execute the pipeline with the id of 01 but with my yaml override and return the expanded yaml pipeline.

Preview - Preview Service: Pipelines API Version: 6.1-preview.1 Queues a dry run of the pipeline and returns an object containing the final yaml.

# AzurePipelinesPS session
$session = 'myAPSessionName'

# Path to my local yaml
$path = ".\extension.yml"    

# The id of an existing pipeline in my project
$id = 01        
        
# The master branch of my repository
$resources = @{              
   repositories = @{
       self = @{
           refName = 'refs/heads/master'
        }
   }
}

Test-APPipelineYaml -Session $session -FullName $path -PipelineId $id -Resources 
$resources
2
votes

Such tool does not exists at the moment - There are a couple existing issues in their feedback channels:

As a workaround - you can install azure devops build agent on your own machine, register as its own build pool and use it for building and validating yaml file correctness. See Jamie's answer in this thread

Of course this would mean that you will need to constantly switch between official build agents and your own build pool which is not good. Also if someone accidentally pushes some change via your own machine - you can suffer from all kind of problems, which can occur in normal build machine. (Like ui prompts, running hostile code on your own machine, and so on - hostile code could be even unintended virus infection because of 3rd party executable execution).

As a workaround to this problem - it's possible to use powershell syntax to maximize the functionality done on build side and minimize functionality done on azure devops.

parameters:
  - name: publish
    type: boolean
    default: true

parameters:
  - name: noincremental
    type: boolean
    default: false

...

      - task: PowerShell@2
        displayName: invoke build
        inputs:
          targetType: 'inline'
          script: |
            # Mimic build machine
            #$env:USERNAME = 'builder'

            # Backup this script if need to troubleshoot it later on
            $scriptDir = "$(Split-Path -parent $MyInvocation.MyCommand.Definition)"
            $scriptPath = [System.IO.Path]::Combine($scriptDir, $MyInvocation.MyCommand.Name)
            $tempFile = [System.IO.Path]::Combine([System.Environment]::CurrentDirectory, 'lastRun.ps1')
            if($scriptPath -ne $tempFile)
            {
                Copy-Item $scriptPath -Destination $tempFile
            }

            ./build.ps1 'build;pack' -nuget_servers @{
                'servername' = @{
                    'url' = "https://..."
                    'pat' = '$(System.AccessToken)'
                }

                'servername2' = @{
                    'url' = 'https://...'
                    'publish_key' = '$(ServerSecretPublishKey)'
                }
            } `
            -b $(Build.SourceBranchName) `
            -addoperations publish=${{parameters.publish}};noincremental=${{parameters.noincremental}}

And on build.ps1 then handle all parameters as seems to be necessary.

param ( 

    # Can add operations using simple command line like this: 
    #   build a -add_operations c=true,d=true,e=false -v
    # => 
    #   a c d
    #
    [string] $addoperations = ''
)

...

foreach ($operationToAdd in $addoperations.Split(";,"))
{
    if($operationToAdd.Length -eq 0)
    {
        continue
    }

    $keyValue = $operationToAdd.Split("=")

    if($keyValue.Length -ne 2)
    {
        "Ignoring command line parameter '$operationToAdd'"
        continue
    }

    if([System.Convert]::ToBoolean($keyValue[1]))
    {
        $operationsToPerform = $operationsToPerform + $keyValue[0];
    }
}

This will allow to run all the same operations on your own machine, locally and minimize amount of yaml file content.

Please notice that I have added also last execution .ps1 script copying as lastRun.ps1 file.

You can use it after build if you see some non reproducible problem - but you want to run same command on your own machine to test it.

You can use ` character to continue ps1 execution on next line, or in case it's complex structure already (e.g. @{) - it can be continued as it's.

But even thus yaml syntax is minimized, it still needs to be tested - if you want different build phases and multiple build machines in use. One approach is to have special kind of argument -noop, which does not perform any operation - but will only print what was intended to be executed. This way you can run your pipeline in no time and check that everything what was planned to be executed - will gets executed.

-1
votes

Trying to write an entire YAML file manually in a standard text editor could be cumbersome. Instead of that, there are open source extensions for Azure DevOps operations that could make your life a lot easier. For instance, a friend of mine recently suggested WhiteSource Bolt when I was trying to add a build task to my YAML pipeline, you can easily install for free and configure your pipeline and I felt like it was a lot easier to use Azure DevOps with Whitesource bolt. Now, to add your build task to the YAML pipeline, follow these steps:

  1. In the pipeline edit page, from the right side, click Show assistant. The Tasks sidebar is displayed.
  2. In the search bar, enter WhiteSource. The WhiteSource task is displayed.

Screenshot attached

  1. Click the WhiteSource Bolt task.
  2. From the bottom right corner, click Add. The WhiteSource Bolt task is added to the pipeline.
  • Task: WhiteSource@21
  • Inputs: cwd: '$(System.DefaultWorkingDirectory)'
  1. To specify the WhiteSource project’s name to be created in WhiteSource Essentials, add the following to the WhiteSource task. In the following example, replace New_Project_Name with the name you want to give your WhiteSource project: NOTE: You cannot change the project name after the first build run.
  • Task: WhiteSource@21
  • Inputs: cwd: '$(System.DefaultWorkingDirectory)'
  • ProjectName: New_Project_Name

If there are errors in the YAML, then the extension itself would validate it for you and provide the error details. For more information on how to add the build task to the YAML pipeline, check the link above.