Gitlab CI pipeline to run jobs parallel in same stage and invoke/trigger other jobs of same stage

Question

I am trying to create a automation pipeline for data load. I have a scenario as explained below:

stages
 - stage1
- stage2

job1:
  stage: stage1
  script:
  - echo "stage 1 job 1"


job2:
  stage: stage1
  script:
   - echo "stage 1 job 2"

job3:
  stage: stage1
  script:
   - echo "stage 1 job 3"

job4:
  stage: stage1
  script:
   - echo "stage 1 job 4"

I want to run the job1 and job2 parallel in the same stage. So, after Job1 and job2 success

job1 will invoke/trigger the job3. that means job3 will start automatically when job1 successes
job2 will invoke/trigger the job4 that means job4 will start automatically when job2 successes

I am writing the pipeline in the .gitlab-ci.yml.

Can anyone help me to implement this?

Mafor Mafor · Accepted Answer · 2020-10-12T23:49:33

Strict implementation of your requirements is not possible (according to my knowledge), the jobs 3 and 4 would need to be in a separate stage (although support for putting them in the same stage is planned). To be clear: the other functional requirements can be fulfilled, i.e:

job1 and job2 start in parallel
job1 will trigger the job3 (immediately, without waiting for the job2 to finish)
job2 will trigger the job4 (immediately, without waiting for the job1 to finish)

The key is using needs keyword to convert the pipieline to a directed acyclic graph:

stages:
    - stage-1
    - stage-2

job-1:
    stage: stage-1
    needs: []
    script: 
      - echo "job-1 started"
      - sleep 5
      - echo "job-1 done"

job-2:
    stage: stage-1
    needs: []
    script: 
      - echo "job-2 started"
      - sleep 60
      - echo "job-2 done"

job-3:
    stage: stage-2
    needs: [job-1]
    script: 
      - echo "job-3 started"
      - sleep 5
      - echo "job-3 done"

job-4:
    stage: stage-2
    needs: [job-2]
    script: 
      - echo "job-4 started"
      - sleep 5
      - echo "job-4 done"

As you can see on the screenshot, the job 3 is started, even though the job 2 is still running.

Gitlab CI pipeline to run jobs parallel in same stage and invoke/trigger other jobs of same stage

1 Answers