Azure DevOps Pipeline with AWS CodeDeploy – Copy appspec.yml and hooks scripts to the same location as the build artifact

Question

(Sorry for the long post! I wanted to put in as much detail as possible)

Using Azure DevOps I am trying to deploy using AWS CodeDeploy. I already have a successful AWS pipeline that utilizes AWS CodeBuild and AWS CodeDeploy So I know everything works in that environment. My organization now wants to convert some of our processes to use Azure DevOps.

In Azure I have successfully built the code into a war file that will be used in the deployment process. This proves that he azure-pipelines.yml file is set up correctly to build. Just to get familiar with Azure DevOps talking to AWS I was able to also upload the war artifact to S3 using the Azure releases process and a properly configured S3 agent. I now want deploy the war artifact directly from Azure to AWS CodeDeploy. Just to ensure that I can interface with AWS CodeDepoy from Azure I have successfully had the war artifact copied into the AWS CodeDeploy application/deployment group. It did copy it over but it fails to deploy because it does not know how to get to the AWS CodeDeploy appspec.yml and the hooks scripts outlined in the appspec.yml.

I have the hooks located in my code project under /aws

I have the following:
/aws/before_install.sh
/aws/after_install.sh
/aws/application_start.sh
/aws/application_stop.sh
/aws/before_allow_traffic.sh
/aws/after_allow_traffic.sh

The appspec.yml I placed in the root directory in my code project:

 /appspec.yml.

I assume that I need to have the appspec.yml and all the shell scripts in “/aws” copied to the same location that the war file is being copied to (I assume that this would be the Revision Bundle location). I observed that the Azure Release process/CodeDeploy agent looks at the “Revision Bundle” location, zips all the files it finds there and passes that to AWS CodeDeploy.

What do I need to add to the azure-pipelines.yml to tell it to copy the appspec.yml and all the shell scripts in /aws to the same location that the war file is placed under and make this the Revision Bundle location?

Using the below azure-pipelines.yml it builds the war and as an experiment I do see that it does copy the one shell script I defined because I do see it stating that two artifacts were uploaded (pro-0.0.1-SNAPSHOT.war and after_install.sh) when I run it :

trigger:
- master

pool:
  vmImage: 'ubuntu-latest'

steps:
- task: Gradle@2
  inputs:
    workingDirectory: ''
    gradleWrapperFile: 'gradlew'
    gradleOptions: '-Xmx3072m'
    javaHomeOption: 'JDKVersion'
    jdkVersionOption: '1.11'
    jdkArchitectureOption: 'x64'
    publishJUnitResults: true
    testResultsFiles: '**/TEST-*.xml'
    tasks: 'build'
- publish: $(System.DefaultWorkingDirectory)/build/libs/
  artifact: pro-0.0.1-SNAPSHOT.war

- task: CopyFiles@2
  displayName: 'Copy the needed AWS files to the artifact directory'
  inputs:
    SourceFolder: '$(build.artifactstagingdirectory)'
    Contents: 'aws/*.sh'
    TargetFolder: '$(System.DefaultWorkingDirectory)/build/libs/'
- publish: $(System.DefaultWorkingDirectory)/aws/after_install.sh
  artifact: after_install.sh

Now how do I get all the shell scripts uploaded? I tried wildcards but it didn’t work.

How do I have the Azure Codedeploy agent access/find all the files together to allow them to be zipped up and sent along to AWS CodeDeploy?

To summarize - I need all of the following files in one directory so that the Azure Codedeploy agent can access them to zip up and send to AWS CodeDeploy: pro-0.0.1-SNAPSHOT.war, appspec.yml, after_install.sh, before_install.sh, application_start.sh, application_stop.sh¸ after_allow_traffic.sh, before_allow_traffic.sh,

FYI – I have the “Revision Bundle” location set to “$(System.DefaultWorkingDirectory)/_Trove /pro-0.0.1-SNAPSHOT.war/

Thanks for any help or advice!!

Answer 1

Try with below YAML script. Just do little changes to your CopyFiles and second Publish task:

trigger:
- master

pool:
  vmImage: 'ubuntu-latest'

steps:
- task: Gradle@2
  inputs:
    workingDirectory: ''
    gradleWrapperFile: 'gradlew'
    gradleOptions: '-Xmx3072m'
    javaHomeOption: 'JDKVersion'
    jdkVersionOption: '1.11'
    jdkArchitectureOption: 'x64'
    publishJUnitResults: true
    testResultsFiles: '**/TEST-*.xml'
    tasks: 'build'
- publish: $(System.DefaultWorkingDirectory)
  artifact: pro-0.0.1-SNAPSHOT.war

- task: CopyFiles@2
  displayName: 'Copy the needed AWS files to the artifact directory'
  inputs:
    SourceFolder: '$(System.DefaultWorkingDirectory)'
    Contents: |
      aws/*.sh
      appspec.yml
    TargetFolder: '$(build.artifactstagingdirectory)'
- task: PublishBuildArtifacts@1
  inputs:
    PathtoPublish: '$(Build.ArtifactStagingDirectory)'
    ArtifactName: 'build.gradle'
    publishLocation: 'Container'

Answer 2

For others that might have a similar situation, here is what finally worked for me :

trigger:
- '*'
name: $(SourceBranchName)_$(Year:yy).$(Month).$(DayOfMonth)$(Rev:.r)-$(BuildId)

pool:
  vmImage: 'ubuntu-latest'

steps:
- task: Gradle@2
  inputs:
    workingDirectory: ''
    gradleWrapperFile: 'gradlew'
    gradleOptions: '-Xmx3072m'
    javaHomeOption: 'JDKVersion'
    jdkVersionOption: '1.11'
    jdkArchitectureOption: 'x64'
    publishJUnitResults: true
    testResultsFiles: '**/TEST-*.xml'
    tasks: 'build'
# - publish: $(System.DefaultWorkingDirectory)
#   artifact: pro-0.0.1-SNAPSHOT.war

- task: CopyFiles@2
  displayName: 'Copy the needed AWS files to the artifact directory'
  inputs:
    SourceFolder: '$(System.DefaultWorkingDirectory)'
    Contents: |
      aws/*.sh
      aws/appspec.yml
      build/libs/*
    TargetFolder: '$(build.artifactstagingdirectory)/output'
    flattenFolders: true 
- task: PublishBuildArtifacts@1
  inputs:
    PathtoPublish: '$(Build.ArtifactStagingDirectory)/output'
    ArtifactName: 'pro'
    publishLocation: 'Container'