Accessing Docker Images In A Multi Stage Azure Devops

Question

I am building an Azure DevOps pipeline and was trying out the multi-stage feature, this is defined by using a yml file.

In the yml definition I have two stages, one is to build docker images using a docker-compose command, second stage is to push these images to ACR. It seems this is not possible as I haven't had any success accessing the recently built images from the first stage. Here's a sample yml file

stages:
- stage: Build
  displayName: Build image
  jobs:  
  - job: Build
    displayName: Build
    pool:
      vmImage: 'ubuntu-latest'
    steps:
    - publish:  $(Build.ArtifactStagingDirectory)
      artifact: docker-images
    - task: DockerCompose@0
      inputs:
        containerregistrytype: 'Azure Container Registry'
        azureSubscription: '************'
        azureContainerRegistry: '************'
        dockerComposeFile: '**/docker-compose.yml'
        action: 'Build services'
        additionalImageTags: '$(Build.BuildId)'
- stage: Push
  displayName: Push image
  jobs:  
  - job: Push
    displayName: Push
    pool:
      vmImage: 'ubuntu-latest'
    steps:
    - download: current
      artifact: docker-images
    - task: DockerCompose@0
      inputs:
        containerregistrytype: 'Azure Container Registry'
        azureSubscription: '************'
        azureContainerRegistry: '************'
        dockerComposeFile: '**/docker-compose.yml'
        action: 'Push services'
        additionalImageTags: '$(Build.BuildId)'

The question is, how do I access docker images that was built in my previous stage? where is it stored? I've tried by downloading $(Build.ArtifactStagingDirectory) from the first stage but it didn't seem to have it. The same applies if I have one stage but separate jobs. If I would to use both build and push in one stage it works fine, but I want to have a separate stages for each.

Levi Lu-MSFT Levi Lu-MSFT · Accepted Answer · 2020-04-14T03:13:38

First of all you should always put publish artifacts task at the end of the stage. Or you will just publish an empty folder.

Second, the dock compose command build and save the image in the docker folder on the hosted machine. Nothing will be output to the artifacts folder $(Build.ArtifactStagingDirectory) of the agent.

As a workaround to pass the docker image between stages, you can use docker image save command to specifically save the image in folder $(Build.ArtifactStagingDirectory). And use publish artifacts task to publish the image to azure devops server. Then you can use download artifacts to download the image in the next stage.

You can check below example:

1, In Build Stage add a Docker@0(version 0.*) after DockerCompose task to run image save command to save the image to folder $(Build.ArtifactStagingDirectory)

- task: Docker@0
  displayName: 'Run a Docker command'
  inputs:
    containerregistrytype: 'Azure Container Registry'
    azureSubscription: '************'
    azureContainerRegistry: '************'
    action: 'Run a Docker command'
    customCommand: 'image save <imageName>:$(Build.BuildId) -o $(Build.ArtifactStagingDirectory)/imagetest.tar'

2, Put the publish artifact task at the end of Build stage to publish the image

- publish:  $(Build.ArtifactStagingDirectory)
  artifact: docker-images

3, Now you can download the image archive file from Build stage to Publish stage. And you can run docker load command to load the archive image. After it is loaded, you can then push it to ACR

- download: current
  artifact: docker-images

- task: Docker@0
  displayName: 'Run a Docker command'
  inputs:
    containerregistrytype: 'Azure Container Registry'
    azureSubscription: '************'
    azureContainerRegistry: '************'
    action: 'Run a Docker command'
    customCommand: 'load --input $(Pipeline.Workspace)/docker-images/imagetest.tar'

Hope above helps!

Accessing Docker Images In A Multi Stage Azure Devops

2 Answers