Jenkins pipeline (parallel && dynamically)?

9
votes

Question

I have simple parallel pipeline (see code) which I use together with Jenkins 2.89.2. Additionally I use parameters and now want to be able to in-/decrease the number of deployVM A..Z stages automatically by providing the parameter before job execution.

How can I dynamically build my pipeline by providing a parameter?

Researched so far:

Code

The pseudo code of what I want - dynamic generation:

pipeline {

    agent any

    parameters {
        string(name: 'countTotal', defaultValue: '3')
    }

    stages {

       stage('deployVM') {

        def list = [:]
        for(int i = 0; i < countTotal.toInteger; i++) {
            list += stage("deployVM ${i}") {
                steps {
                    script {
                        sh "echo p1; sleep 12s; echo phase${i}"
                    }

                }
            }
        }

        failFast true
        parallel list
       }

   }

}

The code I have so far - executes parallel but is static:

pipeline {

    agent any
    stages {

       stage('deployVM') {
        failFast true
        parallel {
            stage('deployVM A') {
                steps {
                    script {
                        sh "echo p1; sleep 12s; echo phase1"
                    }

                }
            }
            stage('deployVM B') {
                steps {
                    script {
                        sh "echo p1; sleep 20s; echo phase2"
                    }

                }
            }
        }
       }

   }

}
3
jenkinsjenkins-pipeline
I would suggest to use scripted pipeline, it's way more flexible. – Vitalii Vitrenko
I read the article but do not see how I could apply it here. Can you provide an example? The documentation is really sparse. – lony

3 Answers

15
votes

Although the question assumes using declarative pipeline I would suggest to use scripted pipeline because it's way more flexible.
Your task can be accomplished this way

properties([
    parameters([
        string(name: 'countTotal', defaultValue: '3')
    ])
])

def stages = [failFast: true]
for (int i = 0; i < params.countTotal.toInteger(); i++) {
    def vmNumber = i //alias the loop variable to refer it in the closure
    stages["deployVM ${vmNumber}"] = {
        stage("deployVM ${vmNumber}") {
            sh "echo p1; sleep 12s; echo phase${vmNumber}"
        }
    }
}

node() {
    parallel stages
}

Also take a look at snippet generator which allows you to generate some scripted pipeline code.

1
votes

@Vitalii

I wrote similiar code piece, but unfoutunelty, all three element been loopped all shows the last one, not sure if it had something to do with groovy / jenkinsfile itself, that some clouse / reference went break with wrong usage

my purpose is to distribute tasks to specific work nodes

node_candicates = ["worker-1", "worder-2", "worker-3"]

def jobs = [:]
for (node_name in node_candidates){
    jobs["run on $node_name"] = {             // good
        stage("run on $node_name"){           // all show the third 
            node(node_name){                  // all show the third 
                print "on $node_name"
                sh "hostname"
            }
        }
    }
}    
parallel jobs

it went totally Ok if I expand / explain the loop, instead of loop over it, like

parallel worker_1: {
    stage("worker_1"){
        node("worker_1"){
            sh """hostname ; pwd """
            print "on worker_1"
        }
    }
},  worker_2: {
    stage("worker_2"){
        node("worker_2"){
            sh """hostname ; pwd """
            print "on worker_2"
        }
    }
},  worker_3: {
    stage("worker_3"){
        node("worker_3"){
            sh """hostname ; pwd """
            print "on worker_3"
        }
    }
}
1
votes

Using Declarative pipeline also you can achieve this.

Follow my answer HERE

In above link answer I have used Var.collectEntries but map also can be used.