Jenkins triggering dependent builds only when necessary?

2
votes

I have the following build scenario:

Project C depends on Project B which depends on Project A

Project X depends on C, Project Y depends on B, Project Z depends on A.

There are many other modules with dependencies, but the most "complex" build relationship is with project X. To set this up in Jenkins, I've used the Parameterized Trigger Plugin, with the "block until triggered project has finished building" option. Each buildable module gets its own Jenkins job, and the plugin is used to block on the necessary dependent build job.

I've run into 2 problems with this setup.

  1. Projects A and B get built many times over as they are in the transitive dependency chain for projects X, Y, Z, etc.
  2. Jenkins, sometimes, gets completely blocked building projects because builds for jobs X, Y, Z, etc. have filled up all available executors (being triggered by scm changes), and the dependent projects are in the queue waiting to be built.

I'm looking for advice on how to configure Jenkins for such a build environment. I'm new to Jenkins so I don't know exactly what options are available to resolve this issue.

EDIT:

All jobs are triggered by SCM changes. Dependent projects are also triggered by the Parameterized Trigger Plugin; the "block until triggered projects finish their builds" options.

1
antjenkins
Can you list out what mechanism(s) you're using to trigger each build - SCM change, dependency change using something like the Ivy plugin, triggered by upstream build, etc ? If some of the builds are triggered by multiple mechanisms, you can probably simplify the triggers. – gareth_bowles
@gareth_bowles Edited question. – Josiah

1 Answers

3
votes

Jenkins has multiple ways of triggering builds. Most common is by watching for repository changes.

However, you can also automatically trigger a build after another job finishes building. This is built into Jenkins. Take a look under Build Triggers and select the checkbox Build after other projects are built, or in the Post-Build section, select Build Other Projects post-build action. And, you can have multiple triggers on a build, so a build could happen if a dependency from another project changes or if the source files change.

If you use Maven or Ivy, there are plugins that can be used to trigger a build if a Maven/Ivy jar dependency that the project uses changes.

One more useful plugin is the Copy Artifacts Plugin. This allows you to copy a build artifact from Project "X" to Project "Y" for Project "Y" to use for building.