Azure pipeline service container - image from AWS ECR

Question

I've been trying to pull a service container from AWS ECR in azure pipeline but I'm not sure how to perform the "aws ecr get-login" from the pipeline.yml. Here is what I have in my azure pipeline.yml but of course I'm getting "no basic auth credentials" error. Can anyone shed some lights on how to pull image from AWS ECR in azure pipeline service container ?

resources:
  containers:
  - container: sqlDB
    image: 1511260612345.dkr.ecr.ap-southeast-2.amazonaws.com/sqlDB:latest
    options: --name myDB
    env:
      ACCEPT_EULA: Y
      SA_PASSWORD: myPass123!

services:
  sql_db: sqlDB

I have also tried to connect to the AWS endpoint but stupidly azure pipeline only allows docker registry and its own azure container registry.

"The pipeline is not valid. Expected 'dockerregistry' service connection type for image registry referenced by sqlDB, but got AWS for service connection aws_test."

Answer 1

Your pipeline is correct and what mentioned by @4c74356b41 by adding "endpoint" is also correct.

Following is the Step by step instructions:

1. Assuming you have AWS access and secret key, you need to create profile credential or you can do "aws configure". After that execute the following command:

aws ecr get-login --no-include-email --region [enter you region here]

or if you have aws profile

aws ecr get-login --no-include-email --profile [enter your aws profile] --region [enter you region here]

The copy the password section (after "-p") from the above output (you will need to paste it in the docker registry below).

2. Go to project settings -> Service Connections -> select "Docker Registry" from the new service connection dropdown

3. Enter the detail as follow:

4. set your endpoint in the pipeline.yml to "aws_test"

Answer 2

Just an update to the above answer what you can do to get an image from AWS ECR and download it to Azure Pipeline Executor:

1: You can use aws-vsts-tools for that purpose, all you will need to do it add the aws-vsts extension from Azure MarketPlace and then create a Service Connection with the appropriate permissions:

                "ecr:GetDownloadUrlForLayer",
                "ecr:BatchGetImage",
                "ecr:ListImages",
                "ecr:GetRepositoryPolicy",
                "ecr:DescribeImages",
                "ecr:GetAuthorizationToken",
                "ecr:ListTagsForResource",
                "ecr:UploadLayerPart",
                "ecr:PutImage",
                "ecr:UntagResource",
                "ecr:CompleteLayerUpload",
                "ecr:TagResource",
                "ecr:DescribeRepositories",
                "ecr:InitiateLayerUpload",
                "ecr:BatchCheckLayerAvailability"

Then you can run the docker build step task available in azure pipeline tasks where it will download the image and then you can execute it via your above task.

2: The other way would involve totally bash shell where you will have to run a shell exec task in azure pipeline. Here you will have to run

Docker login

aws ecr get-login-password --region xxx | docker login --username xxx --password-stdin account-number.dkr.ecr.regionxxx.amazonaws.com/repo-name

Docker Pull

docker pull account-number.dkr.ecr.regionxxx.amazonaws.com/repo-name:tag   

Answer 3

The majure issue is that aws ecr get-login-password token is valid only for 12 hours. That limits the use of listed solutions.

One of the option is to use aws-toolkit-azure-devops and its dedicated ECRPullImage task

  - job: ecrPull
    steps: 
    - task: ECRPullImage@1
      inputs:
        awsCredentials: '<AWSServiceConnection>' #Name of the AWS service connection
        regionName: '<region>'
        repository: '<repository>'
        imageSource: 'imagetag'
  - job: Build
    dependsOn: ecrPull
    condition: succeeded()
    container: 
      image: <accountId>.dkr.ecr.<region>.amazonaws.com/<repository>:<tag>