GitLab CI: how can I set two dependencies pipelines to run in parallel?

Question

At the moment, I have a pipeline defined with 4 different stages to build dependencies for my project:

stages:
  - dependencies-A
  - dependencies-B
  - dependencies-C
  - dependencies-D
  - build

Thus, the jobs in the pipeline now run like this:

A -> B -> C -> D -> build

Actually, B depends on A, and D depends on C. The sub-pipelines A->B and C->D being independent, I would like to run those two sub-pipelines in parallel, to speed up the build. So, that way:

A -> B \
         build
C -> D /

Is that possible at all?

Thanks for your kind help.

Answer 1

If you are using Gitlab version > 12.2, you can use needs keyword to create a directed acyclic graph.

In this case your .gitlab-ci.yml shoul look something like this:

Here depB will start after depA successfully completes. Same goes for depC and depD stages. build stage will run after all the other stages are done.

stages:
  - dependencies-A-C
  - dependencies-B-D
  - build

depA:
  stage: dependencies-A-C

depB:
  stage: dependencies-B-D
  needs: ["depA"]

depC:
  stage: dependencies-A-C

depD:
  stage: ependencies-B-D
  needs: ["depC"]

build:
  stage: build

Find more in docs:
https://docs.gitlab.com/ee/ci/directed_acyclic_graph/

https://docs.gitlab.com/ee/ci/yaml/README.html#needs

Answer 2

Not only can you use needs, as detailed in Murli's answer, but with GitLab 13.12 (May 2021), you can see those pipelines, and visually confirm they will run in parallel.

Show job dependencies in the pipeline graph

The full pipeline graph provides a visual representation of all the jobs in your CI/CD pipeline, grouped by stage.
This visualization helps you track the progress of your jobs and sets expectations for the order in which jobs will execute.

The introduction of the needs keyword let you create a directed acyclic graph that lets jobs start running earlier than they would if they were configured solely by stage.

However, this creates ambiguity when looking at the pipeline graph, as its harder to tell in what order you could expect jobs to run.

In this release, you now have the ability to view pipelines by job dependencies when the job order reflects the needs: relationships between jobs.

We’ve also added links between jobs to see exactly which jobs need to complete before a particular job can start. It will be much easier for you to track and understand job dependencies and expected run order when using the needs: keyword.

See Documentation and Epic.