Jenkins parameterized job that only queues one build

Question

Imagine a Jenkins job A which takes 1 minute to run, and job B which takes 5 minutes.

If we configure job A to trigger job B, while job B is running job A may run 5 times before B completes. However, Jenkins doesn't add 5 builds to job B's queue, which is great because otherwise speedy job A would be creating an ever-growing backlog of builds for poor slow job B.

However, now we want to have job A trigger B as a parameterized job, using the parameterized trigger plugin. Parameterized jobs do queue up a backlog, which means job A is happily creating a huge pile of builds for job B, which can't possibly keep up.

It does make sense to add a new parameterized build to the queue each time it's triggered, since the parameters may be different. Jenkins should not always assume that a new parameterized build renders previously queued ones unnecessary.

However, in our case we actually would like this. Job A builds and packages our application, then Job B deploys it to a production-like environment and runs a heavier set of integration tests. We also have a build C which deploys to another environment and does even more testing, so this is an escalating pattern for us.

We would like the queue for our parameterized job B to only keep the last build added to it; each new build would replace any job currently in the queue.

Is there any nice way to achieve this?

Answer 1

Add a "System Groovy Script" pre-build step to job B that checks for (newer) queued jobs of the same name, and bails out if found:

def name = build.properties.environment.JOB_NAME
def queue = jenkins.model.Jenkins.getInstance().getQueue().getItems()
if (queue.any{ it.task.getName() == name }) {
  println "Newer " + name + " job(s) in queue, aborting"
  build.doStop()
} else {
  println "No newer " + name + " job(s) in queue, proceeding"
}

Answer 2

You could get rid of Parameterized Trigger Plugin, and instead, use the traditional triggering. As you said, this would prevent job B queue from piling up.

How to pass the parameters from A to B then? Make job A to yield the parameters in it's console output. In job B, to get these build parameters, examine the console output of the latest A build (with a Python script, perhaps?).

Answer 3

Here's one workaround:

• Create a job A2B between jobs A and B
• Add a build step in job A2B that determines whether B is running. To achieve that, check:
  • Determine if given job is currently running using Hudson/Jenkins API
  • Python API's is_queued_or_running()
• Finally, trigger job B from A2B only if there are no B builds queued or running (carrying the parameters through)

Answer 4

In case you're using Git, this is now supported by the "Combine Queued git hashes" under the Triggering/ Parameters/ Pass-through option. The first Git plugin version that should actually work with this is 1.1.27 (see Jenkins-15160)

Answer 5

Ron's solution worked for me. If you don't like having bunch of cancelled builds in build history you can add the following system groovy script to job A before you trigger job B:

import hudson.model.*  
def q = jenkins.model.Jenkins.getInstance().getQueue()   
def items = q.getItems()  
for (i=0;i<items.length;i++){  
  if(items[i].task.getName() == "JobB"){  
    items[i].doCancelQueue()
  }   
}

Answer 6

Here's a more flexible option if you are only care about a few parameters matching. This is especially helpful when a job is triggered externally (i.e. from GitHub or Stash) and some parameters don't need a separate build.

If the checked parameters match in both value and existence in both the current build and a queued build, the current build will be aborted and the description will show that a future build contains the same checked parameters (along with what they were).

It could be modified to cancel all other queued jobs except the last one if you don't want to have build history showing the aborted jobs.

    checkedParams = [ 
    "PARAM1",
    "PARAM2",
    "PARAM3",
    "PARAM4",
]

def buildParams = null
def name = build.project.name
def queuedItems = jenkins.model.Jenkins.getInstance().getQueue().getItems()

yieldToQueuedItem = false
for(hudson.model.Queue.Item item : queuedItems.findAll { it.task.getName() == name }) {
    if(buildParams == null) {    
        buildParams = [:]
        paramAction = build.getAction(hudson.model.ParametersAction.class)
        if(paramAction) {
            buildParams = paramAction.getParameters().collectEntries {
                [(it.getName()) : it.getValue()]
            }
        }
    }
    itemParams = [:]
    paramAction = item.getAction(hudson.model.ParametersAction.class)
    if(paramAction) {
        itemParams = paramAction.getParameters().collectEntries {
            [(it.getName()) : it.getValue()]
        }
    }

    equalParams = true
    for(String compareParam : checkedParams) {
        itemHasKey = itemParams.containsKey(compareParam)
        buildHasKey = buildParams.containsKey(compareParam)
        if(itemHasKey != buildHasKey || (itemHasKey && itemParams[compareParam] != buildParams[compareParam])) {
            equalParams = false
            break;
        }
    }
    if(equalParams) {
        yieldToQueuedItem = true
        break
    }
}

if (yieldToQueuedItem) {
    out.println "Newer " + name + " job(s) in queue with matching checked parameters, aborting"
    build.description = "Yielded to future build with:"
    checkedParams.each {
        build.description += "<br>" + it + " = " + build.buildVariables[it]
    }

    build.doStop()
    return
} else {
    out.println "No newer " + name + " job(s) in queue with matching checked parameters, proceeding"
}

Answer 7

The following is based on Ron's solution, but with some fixes to work on my Jenkins 2 including removing java.io.NotSerializableException exception and handling that the format of getName() is some times different from that of JOB_NAME

// Exception to distinguish abort due to newer jobs in queue
class NewerJobsException extends hudson.AbortException {
    public NewerJobsException(String message) { super(message); }
}

// Find jenkins job name from url name (which is the most consistently named
// field in the task object)
// Known forms:
//   job/NAME/
//   job/NAME/98/
@NonCPS
def name_from_url(url)
{
    url = url.substring(url.indexOf("/") + 1);
    url = url.substring(0, url.indexOf("/"));
    return url
}

// Depending on installed plugins multiple jobs may be queued. If that is the
// case skip this one.
// http://stackoverflow.com/questions/26845003/how-to-execute-only-the-most-recent-queued-job-in-jenkins
// http://stackoverflow.com/questions/8974170/jenkins-parameterized-job-that-only-queues-one-build
@NonCPS
def check_queue()
{
    def name = env.JOB_NAME
    def queue = jenkins.model.Jenkins.getInstance().getQueue().getItems()
    if (queue.any{ name_from_url(it.task.getUrl()) == name }) {
        print "Newer ${name} job(s) in queue, aborting"
        throw new NewerJobsException("Newer ${name} job(s) in queue, aborting")
    } else {
        print "No newer ${name} job(s) in queue, proceeding"
    }
}