Copy artifacts of multiple builds on same job in Jenkins

Question

I'm using MultiJob plugin and have a job (Job-A) that triggers Job-B several times. My requirement is to copy some artifact (xml files) from each build.

The difficulty I have is that using Copy Artifact Plugin with "last successful build" option will only take the last build of Job-B, while I need to copy from all builds that were triggered on the same build of Job-A

The flow looks like: Job-A starts and triggers:

`Job-A` -->
   Job-B build #1
   Job-B build #2
   Job-B build #3
   ** copy artifcats of all last 3 builds, not just #3 **

Note: Job-B could be executed on different slaves on the same run (I set the slave to run on dynamically by setting parameter on upstream job-A)

When all builds are completed, I want Job-A to copy artifact from build #1, #2 and #3 , and not just from last build. How can I do this?

Answer 1

Here is more generic groovy script; it uses the groovy plugin and the copyArtifact plugin; see instructions in the code comments.

It simply copies artifacts from all downstream jobs into the upstream job's workspace.

If you call the same job several times, you could use the job number in the copyArtifact's 'target' parameter to keep the artifacts separate.

// This script copies artifacts from downstream jobs into the upstream job's workspace.
//
// To use, add a "Execute system groovy script" build step into the upstream job
// after the invocation of other projects/jobs, and specify
// "/var/lib/jenkins/groovy/copyArtifactsFromDownstream.groovy" as script.

import hudson.plugins.copyartifact.*
import hudson.model.AbstractBuild
import hudson.Launcher
import hudson.model.BuildListener
import hudson.FilePath

for (subBuild in build.builders) {
  println(subBuild.jobName + " => " + subBuild.buildNumber)
  copyTriggeredResults(subBuild.jobName, Integer.toString(subBuild.buildNumber))
}

// Inspired by http://kevinormbrek.blogspot.com/2013/11/using-copy-artifact-plugin-in-system.html
def copyTriggeredResults(projName, buildNumber) {
   def selector = new SpecificBuildSelector(buildNumber)

   // CopyArtifact(String projectName, String parameters, BuildSelector selector,
   // String filter, String target, boolean flatten, boolean optional)
   def copyArtifact = new CopyArtifact(projName, "", selector, "**", null, false, true)

   // use reflection because direct call invokes deprecated method
   // perform(Build<?, ?> build, Launcher launcher, BuildListener listener)
   def perform = copyArtifact.class.getMethod("perform", AbstractBuild, Launcher, BuildListener)
   perform.invoke(copyArtifact, build, launcher, listener)
}

Answer 2

I suggest you the following approach:

1. Use Execute System Groovy script from Groovy Plugin to execute the following script:

   import hudson.model.*
   
   // get upstream job
   def jobName = build.getEnvironment(listener).get('JOB_NAME')
   def job = Hudson.instance.getJob(jobName)
   def upstreamJob = job.upstreamProjects.iterator().next()
   
   // prepare build numbers
   def n1 = upstreamJob.lastBuild.number
   def n2 = n1 - 1
   def n3 = n1 - 2
   
   // set parameters
   def pa = new ParametersAction([
     new StringParameterValue("UP_BUILD_NUMBER1", n1.toString()),
     new StringParameterValue("UP_BUILD_NUMBER2", n2.toString()),
     new StringParameterValue("UP_BUILD_NUMBER3", n3.toString())
   ])
   Thread.currentThread().executable.addAction(pa)
   

   This script will create three environment variables which correspond to three last build numbers of upstream job.

2. Add three build steps Copy artifacts from upstream project to copy artifacts from last three builds of upstream project (use environment variables from script above to set build number):

3. Run build and checkout build log, you should have something like this:

   Copied 2 artifacts from "A" build number 4
   Copied 2 artifacts from "A" build number 3
   Copied 1 artifact from "A" build number 2
   

Note: perhaps, script need to be adjusted to catch unusual cases like "upstream project has only two builds", "current job doesn't have upstream job", "current job has more than one upstream job" etc.

Answer 3

You can use the following example from an "Execute Shell" Build Step. Please note it can be run only from the Jenkins Master machine and the job calling this step also triggered the MultiJob.

#--------------------------------------
# Copy Artifacts from MultiJob Project
#--------------------------------------
PROJECT_NAME="MY_MULTI_JOB"
ARTIFACT_PATH="archive/target"
TARGET_DIRECTORY="target"

mkdir -p $TARGET_DIRECTORY
runCount="TRIGGERED_BUILD_RUN_COUNT_${PROJECT_NAME}"

for ((i=1; i<=${!runCount} ;i++))
do
   buildNumber="${PROJECT_NAME}_${i}_BUILD_NUMBER"
   cp $JENKINS_HOME/jobs/$PROJECT_NAME/builds/${!buildNumber}/$ARTIFACT_PATH/* $TARGET_DIRECTORY
done

#--------------------------------------