0
votes

Environment

  • Azure Dev Ops (code repo and pipeline trigger)
  • AWS ECR/ECS (target deployment platform)
  • Docker
  • .NET Core Web application (v5.0)

Current Situation

Presently building the application using dotnet build (powershell script) and pushing the zip file to Azure DevOps artifacts using azurepipeline.yml. This works out fine. I have added another task for ECR Push and that also pushes a generated docker image to the ECR using a Dockerfile in the source code.

Business Problem

We want to be able to chose a specific build (eg 0.1.24) in the Azure Artifact (using a variable to provide version number), and generate a Docker build using the corresponding binaries and the Dockerfile. I am unable to find a way to do so. The specific task is as follows:-

  1. Deploy user updates variable "versionNoToDeploy" with the artifact id or name
  2. Deploy user runs a specific pipeline
  3. Pipeline finds the artifact (assuming its valid, else sends error), unzips the package at temp location (-need help on)
  4. Pipeline runs dockerfile to build the image (-known & working)
  5. Pipeline pushes this image to ECR (-known & working)

The purpose is to keep on building the branch till we get stable builds. This build is deployed on a test server manually and tested. Once the build gets certified, it needs to be pushed to Production ECR/ECS instances.

Our pipeline (specific code only)

- pwsh: ./build.ps1 --target Clean Protolint Compile --runtime $(runtime) 
  displayName: ⚙️ Compile

- task: Docker@2
  displayName: Build
  inputs:
     command: build
     repository: appRepo
     tags: |
       $(Build.BuildId)
       deploy
     addPipelineData: true
     Dockerfile: src\DockerfileNew

- task: ECRPushImage@1
  inputs:
     awsCredentials: 'AWS ECR Connection'
     regionName: 'ap-south-1'
     imageSource: 'imagename'
     sourceImageName: 'myApplication'
     sourceImageTag: 'deploy'
     repositoryName: 'devops-demo'
     outputVariable: 'imageTagOutputVar'

- pwsh: ./build.ps1 --target Test Coverage --skip
  displayName: ???? Test

- pwsh: ./build.ps1 --target BuildImage Pack --runtime $(runtime) --skip
  displayName: ???? Pack

- pwsh: ./build.ps1 --target Publish --runtime $(runtime) --skip
  displayName: ???? Publish

Artifact details

enter image description here

Any specific aspects needed can be provided

2

2 Answers

1
votes

Since it involves manual intervention here, you may consider splitting the workflfow into several jobs like this:

jobs:
- job: BuildAndDeployToTest
  steps:
  - bash: echo "A"

- job: waitForValidation
    displayName: Wait for external validation
    pool: server
    timeoutInMinutes: 4320 # job times out in 3 days
    steps:
    - task: ManualValidation@0
      timeoutInMinutes: 1440 # task times out in 1 day
      inputs:
        notifyUsers: |
          [email protected]
          [email protected]
        instructions: 'Please validate the build configuration and resume'
        onTimeout: 'resume'  


- job: DeployToProd
  steps:
  - bash: echo "B"

This is not exactly what you want in terms of invlving variables, but you will be able to achieve your gaol. Wait for valiadtion and deploy to prod only validated builds.

It rely on ManualValidation task.

Another approach could be using deployment job and approvals:

jobs:
- job: BuildAndDeployToTest
  steps:
  - bash: echo "A"


jobs:
  # Track deployments on the environment.
- deployment: DeployToProd
  displayName: deploy Web App
  pool:
    vmImage: 'Ubuntu-16.04'
  # Creates an environment if it doesn't exist.
  environment: 'PROD'
  strategy:
    # Default deployment strategy, more coming...
    runOnce:
      deploy:
        steps:
        - checkout: self 
        - script: echo my first deployment

For this you need to define evnironment and define approval.

In both ways you will get clear picture what was delivered to prod and information who approved PROD deployment.

1
votes

Finally after playing a lot with the pipeline and custom tweaking the individual steps, I came out with the following (excerpted yml).

This involves having a build version to be stored in a variable, which is referenced in each of the steps of the pipeline. The admin has to decide if they want a general build, producing an artifact; or just deploy a specific build to AWS. The variable having the build-id is evaluated conditionally, and based on that, the steps are executed or bypassed.

- pwsh: ./build.ps1 --target Clean Protolint Compile --runtime $(runtime)
  condition: eq(variables['artifactVersionToPush'], '')
  displayName: ⚙️ Compile

- task: DownloadBuildArtifacts@0
  condition: ne(variables['artifactVersionToPush'], '')
  inputs:
     buildType: 'specific'
     project: 'NitinProj'
     pipeline: 'NitinProj'
     buildVersionToDownload: specific
     buildId: $(artifactVersionToPush)
     downloadType: 'single'
     artifactName: 'app'
     downloadPath: '$(System.ArtifactsDirectory)'   #(this needs to be mentioned as default is Build directory)

 - task: ExtractFiles@1
   displayName: Extract Artifact to temp location
   condition: ne(variables['artifactVersionToPush'], '')
   inputs:
     archiveFilePatterns: '$(System.ArtifactsDirectory)/app/*.zip'    #path need update
     cleanDestinationFolder: false
     overwriteExistingFiles: true
     destinationFolder: src

 - task: Docker@2
   displayName: Docker Build image with compiled code in artifact
   condition: ne(variables['artifactVersionToPush'], '')
   inputs:
     command: build
     repository: myApp
     tags: |
       $(Build.BuildId)
       deploy
     addPipelineData: true
     Dockerfile: src\DockerfileNew

 - task: ECRPushImage@1
   displayName: Push built image to AWS ECR
   condition: ne(variables['artifactVersionToPush'], '')
   inputs:
     awsCredentials: 'AWS ECR Connection'
     regionName: 'ap-south-1'
     imageSource: 'imagename'
     sourceImageName: 'myApp'
     sourceImageTag: 'deploy'
     pushTag: '$(Build.BuildId)'
     repositoryName: 'devops-demo'
     outputVariable: 'imageTagOutputVar'

 - pwsh: ./build.ps1 --target Test Coverage --skip
   condition: eq(variables['artifactVersionToPush'], '')
   displayName: 🚦 Test

 - pwsh: ./build.ps1 --target BuildImage Pack --runtime $(runtime) --skip
   condition: eq(variables['artifactVersionToPush'], '')
   displayName: 📦 Pack

 - pwsh: ./build.ps1 --target Publish --runtime $(runtime) --skip
   condition: eq(variables['artifactVersionToPush'], '')
   displayName: 🚚 Publish

I will update this yml to have steps organized into jobs, but that's an optimization story.. :)