Jenkins pipeline partitioning

Question

We have a multiple jenkins pipeline jobs with steps like:

1. Build -> unit-tests -> push to artifactory
2. Build -> unit-tests -> deploy
3. Build -> unit-tests -> integration tests

etc.

Management wants to unify all that to a one big ass pipeline, and currently my team has 2 approaches how to do it:

a) Create on big ass pipeline job with all the stages inside

The cons of this is that we do not need to deploy or publish to artifactory each single build, so there would be some if statements inside that will skip stages if needed - which will make build history a total mess - because one build can do different thing from another (e.g. build #1 publish binaries, and build #2 run integration tests). Pros is that we have all in one workspace and jenkinsfile.

b) Create a separate job for each unit of task.

Like 'build', 'integration tests', 'publishing' and 'deploying', and then create one orchestrator job that will call smaller jobs in sequence wrapped in stages. Cons of this is that we still have CI spread over different jobs, and artifacts have to be passed in between. Pros, of course, is that we can run them independently if needed, so if you only need unit-tests - you run only unit-tests job, which will also result in normal and meaningful build history.

---

Could you please point out if you would go with a or b, or otherwise how would you do it instead?

Answer 1

If the reason for unifying them is code repetition, look at shared libraries. Your Build and unit-tests which is common to all pipelines can go into the shared library and you can just call the library code from different pipelines.

Answer 2

We have one "big ass pipeline", spiced up with

        stage('Push') {
        when {
            expression { env.PUSH_TO_ARTIFACTORY }
            beforeAgent true
        }
        steps {

etc.

Regarding history, you can change your build description, so for builds that push you can add a * symbol in the end, e.g.

    def is_push = env.PUSH_TO_ARTIFACTORY ? " *" : ""
    currentBuild.displayName += "${is_push}"

Having everything in one file means that you don't need to figure out which file to look at as you fix things.