Jenkins upstream - downstream job - quiet period

Question

Jenkins Scenario details:
======================
- Number of build executors(either on master/slave) in Jenkins: 3
- UpStream job: USJob and this job can run on any build executor
- DownStream job: DSJob & this job has a quiet period of 120 seconds + it's tied to run on a particular build executor only.

• USJob has this in the build step : echo "Happy Birthday James" and it takes 5 seconds to complete this job
• DSJob has this in the build step : echo "James bond is dead" and it takes 5 seconds to complete this job
  
  

Now, let say we run USJob (parent/UpStream job) 5 times, which will ---> call DSJob(child/DownStream job) 5 times as well, then, what I want is:

1. Jenkins should run USJob 5 times, thus call DSJob child job during each call.

2. Instead of running DSJob (as soon as it's called from USJob), DSJob will sit idle or in queue for "120 seconds" (i.e. it's set for quiet period).

3. Now, if we see this scenario, UPJob will call DSJob 5 times and DSJob will sit in queue for until that quite period is met. Thus, once the quiet period is over, Jenkins will start DSJob.

My question: What I'm trying to see is what setting/option can I set in DSJob(child job) so that DSJob runs only once and dont care how many time it was called. In other words: If James Bond/someone is dead once, he can't die again! ......got it! but someone can wish him Happy Birthday N # of times on his BDay.

-- THIS concept is similar to running a continuous integration (CI) build in accumulate fashion in TFS (Team Foundation Server - inside Build Definition's TRIGGER section) i.e. run the build as soon as there's a change in source control BUT accumulate all the changes to source control until the running CI build is in progress and once that's complete, the next CI build will pick all other source control changes done by developers.

Answer 1

I agree as this is one option and I would have gone this way finally. Thanks for sharing Eldad. We basically wanted to not to use by putting a file in the Workspace as we run jobs on any available slave on any machine and didn't want to create the file at a central NAS accessible to all machine/slaves. Also, I didn't wanted to make child/downstream job look for parent/upstream job if it finished with X status or not and then run it.

The way I did it is using Quiet period set to 120 seconds on DSJob + Called "DSJob" from USJob or any other parent of DSJob (you can choose to pass/ or not pass params directly/via a property file) + found that it worked fine. When I scheduled multiple USJob, the first occurrence of USJob called DSJob and it waited for 120 seconds (or X seconds what you want to se), then once USJob first job completed, 2nd USJob started and finished and it called DSJob again but this didn't put a new DSJob in queue albeit it just bumped the remaining X no. of seconds for job DSJob to run from X-whatever time spent so far ..back to X seconds again, which was good. I also used "Build Blocker plugin" but I just used it to make my point clear logically, as things are just working like I want using "quiet period" concept set on DSJob. Solved!

Answer 2

If I understand you correctly, you don't want 5 executions of DSJob to happen after 5 (quick) executions of USJob. You rather want DSJob to execute once, from the last trigger from USJob? Inspired by how the quiet period feature works for SCM?

If that is the case, then I faced the same problem. What I did, was to write a Groovy script that executes as the last step in USJob. First, it gets a list of all downstream jobs (that would simply return DSJob in your case). You could skip this step if you don't need it to work dynamically. If any of these jobs are queued, then the script removes them from the queue. USJob would then trigger DSJob (again) as normal.

The end result is that DSJob will only trigger once if USJob has triggered several times within the 120s time frame. Only the last trigger from USJob will have effect.

I'm kind of a Groovy novice, so my script is a messy combination of other scripts I found around the web. I know there are way better ways to do this. I'll post the (messy) script if this is the solution you're looking for.

Answer 3

I suggest the following idea: Don't link the US and DS jobs at all. Have the US job do its thing and finish. Have the DS job check something to decide if to start at all.

To implement this, use the Script Trigger Plugin. I use it for a similar need and it works great! You have full control on the triggering and using a well written script, you can apply ANY logic you want, giving you absolute control over triggering and flow.

Just a note - the script for evaluating build does not have to be kept as an external file. It can also be written in to the job's configuration. Groovy scripts are also supported.

Hope this helps.