Azure Devops Pipeline: Possible to cache task container?

Question

I'm setting up a multi-stage Azure Devops yaml pipeline for a .Net Framework application.

Part of the pipeline will involve using the AWSPowerShellModuleScript task to configure load balancer rules in AWS.

My Task looks like so...

        - task: [email protected]
          name: SetupLoadBalancerRules
          inputs:
            awsCredentials: 'My AWS Service Connection'
            regionName: 'ap-southeast-2'
            scriptType: 'filepath'
            filePath: 'pipeline-scripts/manage-aws-load-balancer-rules.ps1'

Everything is working correctly. However the AWSPowerShellModuleScript tasks are quite slow to initialise. The powershell itself is very fast, but the task requires approximately 1.5 minutes to setup.

I'm running 2 of these tasks in different stages of my pipeline, so this adds 3 minutes to the total time. This may not seem like a lot, but the application itself is quite small, so the setup for these tasks is actually the most time consuming part of the pipeline.

As far as I can tell, it seems that the pipeline is starting a generic container, and then installing the AWS Powershell tools, every time it needs to run one of these tasks.

This seems to be very wasteful and inefficient, so I was wondering if there might be some better way to handle it, for example, caching the built container after the powershell tools are installed, or use an existing image with the tools already installed etc.

I'm very new to using the yaml pipelines, so I'm not sure what's possible.

I like my pipelines to be as efficient as possible, so it just bothers me that this is re-running this repetetive install process every time I need to run a simple powershell script.

Also I should mention that I'm using a hosted Devops Agent... vmImage: 'windows-2019'

Just in case it helps. This is from the task log output...

Checking install status for AWS Tools for Windows PowerShell module.
AWS Tools for Windows PowerShell module not found.
Installing AWS Tools for Windows PowerShell module to current user scope

Name                           Version          Source           Summary                                               
----                           -------          ------           -------                                               
nuget                          2.8.5.208        https://onege... NuGet provider for the OneGet meta-package manager

So it determines that the AWS Tools are not installed, and then possibly uses nuget to install it??

I thought perhaps I could use a cache task to cache the install, but even if I could find where the tools are installed to, it seems unlikely that simply restoring the folder would be sufficient.

Cece Dong - MSFT Cece Dong - MSFT · Accepted Answer · 2020-08-31T08:16:31

Using a Microsoft-hosted agent, each time you run a pipeline, you get a fresh virtual machine. So the tool needs to be installed in each pipeline.

A stage is one or more jobs, which are units of work assignable to the same machine. Using Microsoft-hosted agent, each stage uses a separate agent generally. So the tool will be installed in each stage.

In a word, Microsoft-hosted agent is not be able to cache tools. In order to pre-install the tool or not install tool every time, you could deploy Self-hosted Windows agents, and install the tool on every machine running agent service.

Azure Devops Pipeline: Possible to cache task container?

1 Answers