Sequential stages within parallel pipeline in Jenkins

Question

I have a dynamic scripted pipeline in Jenkins that has many parallel stages, but within each stage, there are multiple serial steps. I have wasted several days trying to make it work: no matter what I try, all serial substages are lumped into one stage! Here is what I have now:

node () {
    stage("Parallel Demo") {
        // Canonical example to run steps in parallel

        // The map we'll store the steps in
        def stepsToRun = [:]

        for (int i = 1; i < 5; i++) {
            stepsToRun["Step${i}"] = { node {
                echo "start"
                sleep 1
                echo "done"
            }}
        }
        // Actually run the steps in parallel
        // parallel takes a map as an argument
        parallel stepsToRun
    }
}

It gets me this beautiful parallel pipeline:

However, the moment I add a serial stage, aka:

node () {
    stage("Parallel Demo") {
        // Run steps in parallel

        // The map we'll store the steps in
        def stepsToRun = [:]

        for (int i = 1; i < 5; i++) {
            stepsToRun["Step${i}"] = { node {
                stage("1") {
                    echo "start 1"
                    sleep 1
                    echo "done 1"
                }
                stage("2") {
                    echo "start 2"
                    sleep 1
                    echo "done 2"
                }                
            }}
        }
        // Actually run the steps in parallel
        // parallel takes a map as an argument
        parallel stepsToRun
    }
}

I get this ugly thing, which looks exactly the same:

To add to the offense, I see the sub-steps executed. How can I get my sub-steps show up as stages?

Also, if there is a way to have dynamic stages (sequential and parallel) with the declarative pipeline, I'm all for it. I found you can do static sequential stages but have little clue how to make it dynamic without going back to scripted pipelines.

Sers Sers · Accepted Answer · 2020-01-10T23:50:29

Here is how you can do something like you want

def stepsToRun = [:]

pipeline {
    agent none

    stages {
        stage ("Prepare Stages"){
            steps {
                script {
                    for (int i = 1; i < 5; i++) {
                        stepsToRun["Step${i}"] = prepareStage("Step${i}")
                    }   
                    parallel stepsToRun
                }
            }
        }
    }
}

def prepareStage(def name) {
    return {
        stage (name) {
            stage("1") {
                echo "start 1"
                sleep 1
                echo "done 1"
            }
            stage("2") {
                echo "start 2"
                sleep 1
                echo "done 2"
            }
        }
    }
}

Sequential stages within parallel pipeline in Jenkins

1 Answers