How to make Pipeline job to wait for all triggered parallel jobs?

Question

I've Groovy script as part of the Pipeline job in Jenkins as below:

node {
    stage('Testing') {
        build job: 'Test', parameters: [string(name: 'Name', value: 'Foo1')], quietPeriod: 2, wait: false
        build job: 'Test', parameters: [string(name: 'Name', value: 'Bar1')], quietPeriod: 2, wait: false
        build job: 'Test', parameters: [string(name: 'Name', value: 'Baz1')], quietPeriod: 2, wait: false
        build job: 'Test', parameters: [string(name: 'Name', value: 'Foo2')], quietPeriod: 2, wait: false
        build job: 'Test', parameters: [string(name: 'Name', value: 'Bar2')], quietPeriod: 2, wait: false
        build job: 'Test', parameters: [string(name: 'Name', value: 'Baz2')], quietPeriod: 2, wait: false
    }
}

which executes multiple other freestyle jobs in parallel, because of wait flag being set to false. However I'd like for the caller job to finish when all jobs are finished. Currently the Pipeline job triggers all the jobs and finishes it-self after few seconds which is not what I want, because I cannot track the total time and I don't have ability to cancel all triggered jobs at one go.

How do I correct above script for Pipeline job to finish when all jobs in parallel are completed?

I've tried to wrap build jobs in waitUntil {} block, but it didn't work.

Answer 1

You should use pipeline parallel expression, which will wait for all spawned jobs / subtasks to complete:

stage('testing') {
    def branches = [:]

    for(i = 0; i < params.size(); i += 1) {
        def param = params[i]

        branches["Test${i}"] = {
            build job: 'Test', parameters: [string(name: 'Name', value: param)], quietPeriod: 2
        }
    }
    parallel branches
}

You can find some more examples in pipeline docs at jenkins.io

Answer 2

Just run into the same problem, and find a working solution. Just use foreach.

stage('testing') {
    def jobs = [:]

    [1,2,3,4,5].each{
        i -> jobs["Test${i}"] = {
            build job: 'Test', 
            parameters: [string(name: 'theparam', value: "${i}")],
            quietPeriod: 2
        }
    }
    parallel jobs
}

Answer 3

However @agg3l's example is not working with multiple jobs.

Map jobResults = [:]

Boolean failedJobs = false
def buildJobWithParams(def jobs_list, Map results) {
  def branches = [:]    
  for(job in jobs_list)
  {
    print job
    branches["Test-${job}"] = {
       def jobBuild = build job: job, propagate: false
       def jobResult = jobBuild.getResult()
       echo "Build of '${job}' returned result: ${jobResult}"
       results[job] = jobResult
    }
  }    
  return branches
}

stage('Run integration tests') {
      steps {
            def job_branch = buildJobWithParams(item_list, jobResults)
            print job_branch
            parallel job_branch
          }
}

The item_list has more than 1 job, however it will execute only last job multiple times.

Answer 4

This works for me. Triggers 3 jobs. Wait for them to finish. Pay attention to the extra "->" to specify groovy closure. I have one -> on the each loop, and one on the parallel line. It means that the value will be evaluated when running the parallel section.

def jobsString = "job1,job2,job3"
ArrayList jobsList = jobsString.split('\\,')

def parallelJobs2Run = [:]
jobsList.each { job ->
    echo "Going to parallel for job ${job}"
    parallelJobs2Run["${job}"] = { ->
        echo "Calling job ${job}"
        jobResults=build job: "${pathJenkinsFolder}${job}",
        parameters: [
            string(name: 'param1', value: "${value1}"),
            string(name: 'param2', value: "${value2}")
        ],
        propagate: true,
        wait: true

        // List of values: https://stackguides.com/questions/46262862/how-to-i-get-the-url-of-build-triggered-with-build-step-on-jenkins
        buildNumber = ${jobResults.number}
        echo "${job} Build number |${buildNumber}| result: |${jobResults.result}|"
        echo "See details on: |${jobResults.absoluteUrl}|"
    }
};
parallel parallelJobs2Run

Answer 5

Another approach that basically does the same thing is wrapping your jobs in a separate stage and then wrapping all sub-stages in parallel{}, like so:

stage('Trigger all dependent jobs') {
    parallel {
        stage('Trigger task 1') {
            steps {
                build job: "task-1"
            }
        }
        stage('Trigger task 2') {
            steps {
                build job: "task-2"
            }
        }
        stage('Trigger task 3') {
            steps {
                build job: "task-3"
            }
        }
    }   
}

This approach allows you to do whatever you want on each parallel stage - the tasks above may very well be entirely different (from one another) Jenkins jobs.

Also, the above approach (to my knowledge) represents the declarative pipeline syntax.