How to achieve actual parallel execution of jobs on Azure Pipelines on pool of Microsoft-hosted agents?

Question

I have a pretty simple but slow (~15 min) Node test that I want to run on Ubuntu and Linux, and for each on Node 6, 8 and 10 - so 6 "jobs" in total - via an Azure Pipeline on Azure DevOps.

My azure-pipeline.yml looks like this:

jobs:
- job: Ubuntu
  pool:
    vmImage: 'Ubuntu 16.04'
  strategy:
    matrix:
      node_6_x:
        node_version: 6.x
      node_8_x:
        node_version: 8.x
      node_10_x:
        node_version: 10.x
  steps:
  - task: NodeTool@0
    inputs:
      version: $(node_version)
    displayName: 'Install Node.js $(node_version)'  
  - script: |
      npm install
    displayName: 'npm install'
  - script: |
      npm run test
    displayName: 'npm test'
- job: Windows
  pool:
    vmImage: 'vs2017-win2016'
  strategy:
    matrix:
      node_6_x:
        node_version: 6.x
      node_8_x:
        node_version: 8.x
      node_10_x:
        node_version: 10.x
  steps:
  - task: NodeTool@0
    inputs:
      version: $(node_version)
    displayName: 'Install Node.js $(node_version)'  
  - script: |
      npm install
    displayName: 'npm install'
  - script: |
      npm test
    displayName: 'npm test'

As this is an open source repository on GitHub, I would have expected these 6 test runs to happen in parallel (as there should be 10 parallel jobs possible).

Observing the pipeline runs of the pull request adding the azure-pipeline.yml to my repository, it seems like there only sometimes seems to be some parallelism going on. Often I wait minutes for any job to start. Is this maybe a capacity problem on Azure Pipelines side, that no agents to run the tests are available?

When something starts, it is mostly just 1 job per operating system, while the others from the matrix are "Not started / Queued". Shouldn't matrix jobs be executed in parallel?

Which brings me to my real question:
Is there a way to achieve actual parallel execution of jobs on Azure Pipelines on the pools of Microsoft-hosted agents?

janpio janpio · Accepted Answer · 2018-09-17T17:29:11

Changing

  strategy:
    matrix:
      node_6_x:
        node_version: 6.x
      node_8_x:
        node_version: 8.x
      node_10_x:
        node_version: 10.x

to

  strategy:
    maxParallel: 3
    matrix:
      node_6_x:
        node_version: 6.x
      node_8_x:
        node_version: 8.x
      node_10_x:
        node_version: 10.x

(note the additional maxParallel: 3) seems to have done the job: The builds are now started all together as soon as the commit to the PR branch is made.

While maxParallel is currently only documented as "restricts the amount of parallelism.", it seems to be needed to get parallelism for matrix configurations at all.

(Documentation for Job in the YAML schema doesn't help as it shows maxParallel as an alternative to matrix or parallel.)

How to achieve actual parallel execution of jobs on Azure Pipelines on pool of Microsoft-hosted agents?

1 Answers