jenkins pipeline using different nodes for different stages

Question

I am trying to run different stages on different nodes in my jenkins pipeline like stage 1 is checkout, stage 2 is build, stage 3 unit testing. For example :

node ('linux1')
{
stage ('checkout')
..........
}

node ('linux2')
{
stage ('build')
........
}


node ('linux3')
{
stage ('unit testing')
...........
}

If I try to do this, my question is if I checkout code in Linux1 node, how come the other stages can run in different node while the checked out code is in node1.

How the code will be distributed and if my understanding is not correct, how can I achieve parallelism and ensure each stage can parallel and reduce time.

If possible, Please suggest any groovy pipeline documentation for writing Jenkins pipelines.

Code is not going to managed by itself or by tool itself. You have to plan your steps and pipeline in such that code and other resources utilized and available. — Suresh

StephenKing StephenKing · Accepted Answer · 2017-06-11T12:49:52

To transfer the workspace between different nodes, use the stash/unstash.

node('linux1') {
  stage('checkout') {
    checkout(..)
    stash 'name-of-the-stash'
  }
}
node('linux2') {
  stage('build') {
    unstash 'name-of-the-stash'
    sh 'make'
  }
}

Due to this overhead of stashing and unstashing, as well as potentially longer execution times when executors are busy (the next node allocation competes with other builds), I would not recommend excessive node allocations.

Instead, do as much as you can within one node block. You can also mix and match node { stage {..}} and stage { node {..}} within your pipeline codes.

By design (in Jenkins, as well as in the concept of continuous delivery), stages do not execute in parallel. Only the steps within a single stage are executed in parallel.

If you have a stage with long running tasks, use the parallel step. This article could be helpful to further understand this step.

jenkins pipeline using different nodes for different stages

1 Answers