How to run a specific job in gitlab CI

Question

We are facing a problem where we need to run one specific job in gitlab CI. We currently not know how to solve this problem. We have multitple jobs defined in our .gitlab-ci.yml but we only need to run a single job within our pipelines. How could we just run one job e.g. job1 or job2? We can't use tags or branches as a software switch in our environment.

.gitlab-ci.yml:

before_script:
  - docker info

job1:
  script:
    - do something

job2:
  script:
    - do something

Answer 1

You can use a gitlab variable expression with only/except like below and then pass the variable into the pipeline execution as needed.

This example defaults to running both jobs, but if passed 'true' for "firstJobOnly" it only runs the first job.

Old Approach -- (still valid as of gitlab 13.8) - only/except

variables:
  firstJobOnly: 'false'

before_script:
  - docker info

job1:
  script:
    - do something

job2:
  script:
    - do something
  except:
    variables:
      - $firstJobOnly =~ /true/i

Updated Approach - rules

While the above still works, the best way to accomplish this now would be using the rules syntax. A simple example similar to my original reply is below.

If you explore the options in the rules syntax, depending on the specific project constraints there are many ways this could be achieved.

variables:
  firstJobOnly: 'false'

job1:
  script: 
    - do something

job2:
  script: 
    - do something
  rules:
    - if: '$firstJobOnly == "true"'
      when: never
    when: always

Answer 2

We faced the same problem in the past and I'm sharing with you our solution.

#Remark#

• I read the answer of Jawad and I found it good and we have tried it when we faced the issue.
• My remark is that adding when: manual will always show ALL your jobs in the pipeline.
• So if you work in a large team, you can't prevent other collaborators to click by error or by mistake on the job you don't want to be launched.

#What I'm supposing before continuing#

• Let's say that you have 4 jobs.
• You need to always run (manually or automatically) job 1, job 2 and job 4 but NOT job3.
• You want to only run job 3 in a specific case or just when you decide to run it.

#The idea is#

• We launch the 3rd job only for tags which match a regular expression.

• In the example below, it's launched for tags like helloTag.1, helloTag.2, helloTag.3... etc.

• If we are in develop or master (or other branch), we will have 3 stages (stage 1, stage 2, stage 4)

  Note how the 3rd job is not present in the pipeline

  

• Go to "Repository" --> "Tags" --> "New tag"

  Give the tag a name which much your regular expression

  

• If we are in a tag having a name which starts with "helloTag.", we will have 1 stage (stage 3)

  Note how other stages are not present here

  

#Example of .gitlab-ci file#

stages:
    - myStage1
    - myStage2
    - myStage3
    - myStage4

This is my first stage:
    stage: myStage1
    before_script:
        - echo "my stage 1 before script"
    script:
         - echo "my stage 1 script"
    except:
        - /^helloTag.*$/

This is my second stage:
    stage: myStage2
    before_script:
        - echo "my stage 2 before script"
    script:
         - echo "my stage 2 script"
    except:
        - /^helloTag.*$/

This is my third stage:
    stage: myStage3
    before_script:
        - echo "my stage 3 before script"
    script:
         - echo "my stage 3 script"
    only:
        - /^helloTag.*$/

This is my fourth stage:
    stage: myStage4
    before_script:
        - echo "my stage 4 before script"
    script:
         - echo "my stage 4 script"
    except:
        - /^helloTag.*$/

Hope that this helps you.

Answer 3

Simply add a when: manual to the jobs you don't want to run.

These jobs will still appear in your pipeline but won't be run, unless someone "manually" starts them through the web interface, hence the name.

Here's more info about this: https://docs.gitlab.com/ce/ci/yaml/README.html#when

If you're looking for something more "programmable", let's say run either job1 or job2 depending on a branch name or a tag, then you should have a look at the only and except keywords: https://docs.gitlab.com/ce/ci/yaml/README.html#only-and-except

Answer 4

> Currently it seems not to be possible with GitLab CI to have other software switches than tags or branches as provided in the other answers.

We finally switched to an other "real" CI due to too many limitations on GitLab CI. GitLab CI is unfelixble if you want to run some custom jobs in different procedures. I realy appreciated the both answers here. I'm sure they will help other users to manage this stuff. Unfortunately in our case we could not use tags, commit messages or branches as a software switch.

We are still looking for a answer on this. Feel free to give an other approach to solve this problem. I will mark the right answer once it hits. Also a bounty on this question did not result in an right answer.

Answer 5

The original question asks how to trigger jobs without using branch-names or tags. This leaves commit messages and environment variables as viable sources, and neither require editing your yaml file for each push.

Commit Messages

Job rules with commit message regex is a very simple and flexible solution, in my experience.

1. Set up your .gitlab-ci.yml like this

job1:
  ...
  rules:
    - if: '
      $CI_COMMIT_MESSAGE =~ /.*run job1.*/ ||
      $CI_COMMIT_MESSAGE =~ /.*run all.*/
      '

job2:
  ...
  rules:
    - if: '
      $CI_COMMIT_MESSAGE =~ /.*run job2.*/ ||
      $CI_COMMIT_MESSAGE =~ /.*run all.*/
      '

2. Push with a commit message like this

git commit --allow-empty -m "testing conditional job triggers for gitlab-ci based on branch names and commit messages. run job1"
git push

You'll notice this also allows you to run both jobs with a message like this

git commit --allow-empty -m "testing all jobs. run all"
git push

Environment Variables

Elsewhere in your answer to your own question you add a constraint: commit messages cannot be used. Environment variables can be set in the git cli

1. Set up your .gitlab-ci.yml like this

job1:
  ...
  rules:
    - if: $JOB1

job2:
  ...
  rules:
    - if: $JOB2

2. Push like this

git commit --allow-empty -m "triggering job1 with ci variables"
git push -o ci.variable="JOB1=anythingAtAll"

reference: https://docs.gitlab.com/ee/user/project/push_options.html