Adding additional files to Azure Build Pipeline

Question

I have an Azure DevOps Pipeline which pulls from a repository and builds a Visual Studio web project. This is then published to an App Service.

I have several sensitive configuration files which are not included in the repository (or the VS project) and are stored as 'Secure Files' within the Azure DevOps system.

I need to include these files in the 'Config/Secure' folder for the package that gets published (within the zip file). I can download them, but no matter what I try, I cannot get these files to be included in the deployment zip file. They only appear in the 'drop' file system and thus I can't seem to deploy them to the Web App.

Does anyone have any ideas how I can do this? Thanks in advance and Pipeline YAML below:

trigger:
- main

pool:
  vmImage: 'windows-latest'

variables:
  solution: '**/*.sln'
  buildPlatform: 'Any CPU'
  buildConfiguration: 'Release'

steps:
- task: NuGetToolInstaller@1

- task: NuGetCommand@2
  inputs:
    restoreSolution: '$(solution)'

- task: DownloadSecureFile@1
  inputs:
    secureFile: 'AppSettings.secret.config'

- task: DownloadSecureFile@1
  inputs:
    secureFile: 'cache.secret.config'

- task: DownloadSecureFile@1
  inputs:
    secureFile: 'security.secret.config'

- task: DownloadSecureFile@1
  inputs:
    secureFile: 'Smtp.secret.config'

- task: CopyFiles@2
  inputs:
    SourceFolder: '$(Agent.TempDirectory)'
    Contents: |
      AppSettings.secret.config
      cache.secret.config
      security.secret.config
      Smtp.secret.config
    TargetFolder: '$(Build.ArtifactStagingDirectory)/config/secret'
    OverWrite: true
    flattenFolders: true

- task: VSBuild@1
  inputs:
    solution: '$(solution)'
    msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(Build.ArtifactStagingDirectory)\\"'
    platform: '$(buildPlatform)'
    configuration: '$(buildConfiguration)'

- task: VSTest@2
  inputs:
    platform: '$(buildPlatform)'
    configuration: '$(buildConfiguration)'

- task: PublishBuildArtifacts@1
  inputs:
    PathtoPublish: '$(Build.ArtifactStagingDirectory)'
    ArtifactName: 'drop'
    publishLocation: 'Container'

Answer 1

If your project is configured to use them please move them into source control folder not ArtifactStagingDirectory

- task: CopyFiles@2
  inputs:
    SourceFolder: '$(Agent.TempDirectory)'
    Contents: |
      AppSettings.secret.config
      cache.secret.config
      security.secret.config
      Smtp.secret.config
    TargetFolder: '$(Build.ArtifactStagingDirectory)/config/secret'
    OverWrite: true
    flattenFolders: true

So this is wrong because you move it directly to folder which you publish and VSBuild doesn't touch this folder at all.

And if you config folder in on root directory of your repo (and you use here only one repo), this should move your files into solution.

- task: CopyFiles@2
  inputs:
    SourceFolder: '$(Agent.TempDirectory)'
    Contents: |
      AppSettings.secret.config
      cache.secret.config
      security.secret.config
      Smtp.secret.config
    TargetFolder: '$(System.DefaultWorkingDirectory)/config/secret'
    OverWrite: true
    flattenFolders: true

But be aware, that publishing secret files (even as artifact) is not recommended approach. Please consider downloading them and putting into right place just before deploying.

Answer 2

I need to include these files in the 'Config/Secure' folder for the package that gets published (within the zip file)

We can't add new files directly to the zip file. As a workaround, we could Extract the folder $(System.DefaultWorkingDirectory), copy secure file to $(Build.ArtifactStagingDirectory)/PrescQIPPWebApp/config/secret and zip the folder $(Build.ArtifactStagingDirectory)/PrescQIPPWebApp, then publish the Artifact.

In addition, since the zip file will not be deleted after extract, and the PrescQIPPWebApp folder also will not be deleted after archive, we need to add power shell task to delete the zip file and PrescQIPPWebApp folder

I have updated your YAML build definition, you could try it and kindly share the result here.

trigger:
- main

pool:
  vmImage: 'windows-latest'

variables:
  solution: '**/*.sln'
  buildPlatform: 'Any CPU'
  buildConfiguration: 'Release'

steps:
- task: NuGetToolInstaller@1

- task: NuGetCommand@2
  inputs:
    restoreSolution: '$(solution)'

- task: VSBuild@1
  inputs:
    solution: '$(solution)'
    msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(Build.ArtifactStagingDirectory)\\"'
    platform: '$(buildPlatform)'
    configuration: '$(buildConfiguration)'

#Extract PrescQIPPWebApp.zip file to $(Build.ArtifactStagingDirectory)/PrescQIPPWebApp folder
- task: ExtractFiles@1
  inputs:
    archiveFilePatterns: '$(Build.ArtifactStagingDirectory)/PrescQIPPWebApp.zip'
    destinationFolder: '$(Build.ArtifactStagingDirectory)/PrescQIPPWebApp'
    cleanDestinationFolder: false
    overwriteExistingFiles: false

#Delete PrescQIPPWebApp.zip file
- task: PowerShell@2
  inputs:
    targetType: 'inline'
    script: 'Remove-Item ''$(Build.ArtifactStagingDirectory)/PrescQIPPWebApp.zip'''

- task: DownloadSecureFile@1
  inputs:
    secureFile: 'AppSettings.secret.config'

- task: DownloadSecureFile@1
  inputs:
    secureFile: 'cache.secret.config'

- task: DownloadSecureFile@1
  inputs:
    secureFile: 'security.secret.config'

- task: DownloadSecureFile@1
  inputs:
    secureFile: 'Smtp.secret.config'

- task: CopyFiles@2
  inputs:
    SourceFolder: '$(Agent.TempDirectory)'
    Contents: |
      AppSettings.secret.config
      cache.secret.config
      security.secret.config
      Smtp.secret.config
    TargetFolder: '$(Build.ArtifactStagingDirectory)/PrescQIPPWebApp/config/secret'
    OverWrite: true
    flattenFolders: true

#Archive file to PrescQIPPWebApp.zip
- task: ArchiveFiles@2
  inputs:
    rootFolderOrFile: '$(Build.ArtifactStagingDirectory)/PrescQIPPWebApp'
    includeRootFolder: true
    archiveType: 'zip'
    archiveFile: '$(Build.ArtifactStagingDirectory)/PrescQIPPWebApp.zip'
    replaceExistingArchive: true

- task: VSTest@2
  inputs:
    platform: '$(buildPlatform)'
    configuration: '$(buildConfiguration)'

#Delete PrescQIPPWebApp folder
- task: PowerShell@2
  inputs:
    targetType: 'inline'
    script: 'Remove-Item -path ''$(Build.ArtifactStagingDirectory)/PrescQIPPWebApp'' -Recurse -Force -EA SilentlyContinue -Verbose'

- task: PublishBuildArtifacts@1
  inputs:
    PathtoPublish: '$(Build.ArtifactStagingDirectory)'
    ArtifactName: 'drop'
    publishLocation: 'Container'