Jenkins Multibranch Pipeline: clean workspace / delete directory when build ends

4
votes

we are using Jenkins multibranch (scripted) pipeline to run multiple builds, in parallel. Each one will create a new, uniquely named directory to checkout the code and run the test. Directories are uniquely named because they contain the branch name but also a random part like "JAA2A5HUZB7TRE4OTVXU7S...".

This means that after each build X directories (X = number of parallel jobs * number of branches) stay on the Jenkins node, filling disk space.

I wonder how I could automatically delete these directories.

A simplified version of my initial pipeline looks like this

// Initialize the matrix
def matrix = [
  'foo',
  'bar',
]

// Initialize empty tasks map
def tasks = [:]

// Job status
successful = true

// Fill our tasks map from the Matrix data
for (x in matrix) {

  def job_name = x

  tasks[job_name] = {
    node('libvirt') {
        // Checkout repository first
        stage("$job_name - Checkout") {
          checkout scm
        }

        // Then build the machine
        gitlabCommitStatus('build') {
            stage("$job_name - Build") {
                  sh "./bin/build.sh ${job_name}"
            }
        }
      }
    }
  }
}


//// Pipeline ////

notifyBuild('STARTED')

// Run tasks in parallel
try {
  parallel tasks
} catch(e) {
  throw e
} finally {
  if (successful) {
    notifyBuild('SUCCESS')
  } else {
    notifyBuild('FAILED')
  }
}

// Methods
def notifyBuild(String buildStatus = 'STARTED') {
   // used to send formatted e-mails 
}

I first added deleteDir() like this

} finally {
  if (successful) {
    notifyBuild('SUCCESS')
  } else {
    notifyBuild('FAILED')
  }
  node('libvirt') {
    deleteDir()
  }
}

it raises an error and makes the build fail.

I then added a cleanWs() like this

} finally {
  if (successful) {
    notifyBuild('SUCCESS')
  } else {
    notifyBuild('FAILED')
  }
  node('libvirt') {
    cleanWs()
  }
}

The output is

Running on node in /srv/jenkins/workspace/pipeline-deletedir-DRD7EKBEMMWJQZW6KKMQVVBTJTPTKLRAE2ITDK7V7IB5PTFXZUZA
[Pipeline] {
[Pipeline] step
[WS-CLEANUP] Deleting project workspace...[WS-CLEANUP] done
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

but the directory /srv/jenkins/workspace/pipeline-deletedir-DRD7EKBEMMWJQZW6KKMQVVBTJTPTKLRAE2ITDK7V7IB5PTFXZUZA still exists.

I'm looking for any solution to adapt this scripted pipeline (and for declarative pipeline if you know how) to be able to delete all directories created by the build.

4
jenkinsjenkins-pipeline
A partial solution is to leverage the "Orphaned Item Strategy" with aggressive values. This doesn't answer the OP as it only applies to branches that are deleted (and PR branches for closed PR's). But it should help limit the disk space usage to some extent. – Patrice M.
Will "Orphaned Item Strategy" remove the old workspaces for defunct branches across a cluster of remote agents? (including agents off-line when the branch deletion is noticed). – simon.watts

4 Answers

2
votes

First of all, you can not delete the directory in which the build is still running. The deleteDir() step just deletes the content of your workspace.

You could trigger a job that runs after your build is finished which deletes those directories. For this, you could use a pattern like pipeline-deletedir-*.

2
votes

The solution has been to add a try/catch/finally in the stage("$job_name - Build") step, something like

gitlabCommitStatus('build') {
    stage("$job_name - Build") {
        try {
            sh "./bin/build.sh ${job_name}"
         } catch (Exception e) {
            raise e
         } finally {
            deleteDir()
         }
    }
}

This doesn't delete all directories involved (because of this Jenkins issue https://issues.jenkins-ci.org/browse/JENKINS-41805) but it remove all the voluminous directories (containing code/build/artifacts...).

You needs to add "raise e" or build will always return Success, even if one of them fails.

2
votes

I think the best way is adding post section after all stages:

    stages {
         stage("$job_name - Build") {
             sh "./bin/build.sh ${job_name}"
         }
    }
        
    post {
         always {
             echo 'One way or another, I have finished'
             deleteDir() /* clean up our workspace */
         }
    }
0
votes

This doesn't directly answer your question (I personally use deleteDir() and for me it works) but as an alternative to save disk place you could consider discarding "old" builds in your multibranch pipeline:

stage ('Remove old builds') {
    //keep 10 builds per branch
    properties([[$class: 'BuildDiscarderProperty', strategy: [$class: 'LogRotator', artifactDaysToKeepStr: '', artifactNumToKeepStr: '', daysToKeepStr: '', numToKeepStr: '10']]]); 
}