Setup scripts in Azure DevOps MS hosted container pipeline

Question

I need to compile a CMake based C++ project using GCC. It depends on MKL, and for successful Cmake configuration, compilation and test execution, I need run the following comands beforehand

        source /opt/intel/bin/compilervars.sh -arch intel64
        export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH

In order to run this with Azure pipelines I have a container which I am able to run based on the documentation from https://docs.microsoft.com/en-us/azure/devops/pipelines/process/container-phases?view=azure-devops.

Normally the above mentioned setup scripts would be called during container startup (https://hub.docker.com/layers/vvtk/vvcoreazurelinuxdockeragent/latest/images/sha256-c5e3775546ee90a111c9ae700306eb4cd1ebc710686bda5011633c4e5e883e13?context=repo) however it seems (as also described in https://stackoverflow.com/a/63643979/15128314 this startup CMD command is not execute since Azure pipelines do not actually call docker run

As a result I am forced to replicate this into multiple steps of my pipeline job (basically each of config, build and test), since these env vars are also not persistent across the different steps. How can I solve this more efficiently? The pipeline looks horrible ..

  - script: |
      source /opt/intel/bin/compilervars.sh -arch intel64
      export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
      (more cmds here)
    displayName: config_Linux_x64_Release

  - script: |
      source /opt/intel/bin/compilervars.sh -arch intel64
      export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
      (more cmds here)
    displayName: build_Linux_x64_Release
  
  - script: | 
        source /opt/intel/bin/compilervars.sh -arch intel64
        export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
        (more cmds here)
    displayName: test_Linux_x64_Release

Hi Did you get a chance to check out below answer? How did it go? — Levi Lu-MSFT

Levi Lu-MSFT Levi Lu-MSFT · Accepted Answer · 2021-02-08T07:39:46

You can try using the logging command(ie. ##vso[task.setvariable]) to set the system variable and avoid the replication for export command. See below:

You can define environment variable in the pipeline Variable section like below:

variables:
  LD_LIBRARY_PATH: /usr/local/lib:$LD_LIBRARY_PATH

Or you can try adding a script task in the top of your pipeline to run below command:

- script: |
  
    #below script will only take effect in the following tasks. 
      
    source /opt/intel/bin/compilervars.sh -arch intel64
    echo "##vso[task.setvariable variable=LD_LIBRARY_PATH]/usr/local/lib:$LD_LIBRARY_PATH" 

  displayName: 'SetVariable'

Note: the variable set up in above script task only takes effect in the following task.

Please refer to this thread.

Update:

For source command only applies for current shell, and each script step will launch a new shell. So you will still need to repeat source commands in each script step.

Or you can run config, build, test in a single script step to avoid replication.

Setup scripts in Azure DevOps MS hosted container pipeline

1 Answers