Running jobs sequentially in Jenkins

4
votes

Maybe someone could help me. I’d like to run one build at a time in Jenkins for some projects even if there are more builds in the queue. I know that I could limit number of executors but it would affect all projects and this solution doesn’t suit me. I try to use Throttle Concurrent Builds plugin as it’s recommended in this situation but it doesn’t work and I have no idea why.

Before I installed this plugin all builds run correctly. When I have it installed (just installed, I don’t make any changes in configuration) builds go to the queue and never starts (I use 2 executors, both are in idle status all the time). I verified that everything is OK with my node, it’s online, number of executors equals to 2, and it’s set to be utilize as much as possible. There’s no problem with the free space. Increasing number of executors also doesn’t help.

Then I tried to configure Throttle Concurrent Builds (just as an experiment). I went to ‘Configure System’ and add ‘Multi-Project Throttle Category’ with ‘Maximum Total Concurrent Builds’ and ‘Maximum Concurrent Builds Per Node’ set to 1. I went to one of my projects and select ‘Throttle Concurrent Builds’ option. Next, I chose ‘Throttle this project as part of one or more categories’ and checked the check box of the 'Multi-Project Throttle Category' I had just created. As a result when I clicked ‘Build now’ for my project nothing happened. It even didn’t go to the build queue.

Disabling Throttle Concurrent Builds plugin made my projects building again.

I have 2 questions:

  1. Why I couldn’t build any project when I have Throttle Concurrent Builds plugin enabled (builds go to the queue and never start or don’t appear at all)?

  2. Is there any other way to force some jobs running sequentially (also other than changing number of executors)? I have some projects which execute Windows batch command. They must always run separately no matter how many projects I will schedule to build (number of projects and their combination is variable).

Thank you in advance, Anna

PS. I’ve just started my adventure with Jenkins, so please be understanding:)

4
jenkins
There is some good discussion here: stackoverflow.com/questions/11158382/… – Amit
Do you only want a single instance of each job at any one time? Or do you mean you have a list of jobs that you want to run "sequentially" - JobA, then JobB, then JobC, etc? – BitwiseMan
I just remember one plugin Build Flow allows you to define the workflow of the job sequences, sequentially or parallel: wiki.jenkins-ci.org/display/JENKINS/Build+Flow+Plugin – Ranger Wu
Amit -> Thank you, I'll read it. BitwiseMan -> I have list of jobs. Sorry if my description is inaccurate. Ranger Wu -> Thank you, I'll look at this. – Anna

4 Answers

5
votes

If you only want jenkins to run one instance of any job you don't need any special plugin. Simply make sure the "Execute concurrent builds if necessary" checkbox is unchecked for that job.

As the help for that option states:

When this option is checked, multiple builds of this project may be executed in parallel.

By default, only a single build of a project is executed at a time — any other requests to start building that project will remain in the build queue until the first build is complete. This is a safe default, as projects can often require exclusive access to certain resources, such as a database, or a piece of hardware.

But with this option enabled, if there are enough build executors available that can handle this project, then multiple builds of this project will take place in parallel. If there are not enough available executors at any point, any further build requests will be held in the build queue as normal.

This simple "one-or-many" concurrency model is sufficient for a wide range of uses, including the one you've described.

You should only switch to the Throttle Concurrent Builds Plugin if you need more detailed control. For example, "Allow 7 concurrent instances with at most 2 instances per agent (regardless of number of available executors)."

3
votes

The following plugin is for Jenkins and will do what you're looking for:

https://wiki.jenkins-ci.org/display/JENKINS/Multijob+Plugin

Create a 'MultiJob' project, which allows you to define 'phases' for its build.

Many 'phases' can be set up as part of the MultiJob project and each phase "contains" one or more "other" Jenkins jobs. When the MultiJob project is run, the phases will be run sequentially.

Therefore, in order to run N jobs sequentially, add N phases to your MultiJob project, and then add one job to each phase.

Note: There is a setting for each phase that indicates 'Run in parallel' or 'Run sequentially', which I took to mean that it would run the jobs contained in the phase either sequentially or in parallel. However, when I set 'Run sequentially' those jobs still seemed to run in parallel. (I need to investigate this setting further).

1
votes

I found out why Throttle Concurrent Builds plugin didn't work. Version of my Jenkins was really, really old. When I upgrade it to 2.13 projects start to build in expected way and this plugin is exactly what I need. Thank you for help!

0
votes

I used the MultiJob Plugin to do this by adding Build steps.

I used Trigger/call builds on other projects, entered the name of the project to do first, then selected Block until the triggered projects finish their builds. I added a build step for each project I wanted to do in sequence.