Azure Artifacts to download only specific files required for deployment on build server

Question

I have a Package which is prepared manually from selecting the files from Artifacts then download that package and transfer it to a server and then execute powershell scripts from there to deploy. We wanted to implement the deployment via CI pipeline and below are the tasks we are performing created a setup with 2 steps below : Artifacts and Stage The pipeline which has been setup runs fine,downloads all files on build agent server fine and then successfully executes the scripts as required however issue is from the Artifacts section where it is downloading all the other files as well from the drop where we need only 5-6 zip files and rest are not needed. I would want the pipeline to download only the specific folders from the successful build and then execute the scripts from build server. For example below its downloading all but in fact we only need files in folder highlighted in the image :

The pipeline executes sometimes and completes successfully and at time it takes longer or fails with below error message :

2020-09-08T03:36:33.6433775Z 1 downloads remaining.
2020-09-08T03:36:38.6591164Z 1 downloads remaining.
2020-09-08T03:36:43.6665857Z 1 downloads remaining.
2020-09-08T03:36:43.6666024Z ##[warning]No download tasks have completed in 3 minutes. Remaining task statuses:
2020-09-08T03:36:43.6666545Z ##[warning] WaitingForActivation: 1 task(s).
2020-09-08T03:36:48.6822324Z 1 downloads remaining.
2020-09-08T03:36:53.6868753Z 1 downloads remaining.
....

I have tried restarting the VSTS agent service, restart the pipeline but results vary. Could any one please help on how to enable the pipeline to pick only particular folders highlighted in black. Artifacts screen .

For a Given Environment my Pipeline Task has 3 steps :
1.Agent Job
2.Prepare package
3.Deploy

1. Agent Job : Execution Plan :
   Parallelism : None ; Timeout : 0 Job Cancel Time out : 1

2. Prepare Package :
   Script Path : $(System.DefaultWorkingDirectory)/TestRelease2020-1.1/drop/Tools/A/_1.ps1
   Arguments : ".\Tools\Test.Deploy\Test.Deploy.exe"
   ".\Tools\Test.Deploy\Xml\A_PreparePackage.xml" "Source\Project_ABCProject\XYZPackages"
   

   Working Directory : $(System.DefaultWorkingDirectory)/TestRelease2020-1.1/drop
   

3. Deploy : Script Path : $(System.DefaultWorkingDirectory)/TestRelease2020-1.1/drop/Tools/A/_deploy_1.ps1
   Arguments : ".\Tools\Test.Deploy\Test.Deploy.exe"
   ".\Tools\Test.Deploy\Xml\A_config.xml" Working Directory : $(System.DefaultWorkingDirectory)/TestRelease2020-1.1/drop
   

Answer 1

It's by design that the whole build artifact will be downloaded for deployment, we can't control this behavior in release pipeline. Also, it's not necessary to change this behavior in your scenario.

Solution:

To achieve what you want, it's recommended to manage the folders to be published in build pipeline. All we need to do is to make sure the Drop artifact only has the needed folders.

We can use CMD task with xcopy commands to copy the specific folders into the path where we publish the build artifact, so that the Drop artifact only contains the specific folders we need.

Also we can choose ti run CMD task with rmdir(for directories)/del(for files) command to delete/remove the unnecessary files/folders before the final Publish Artifact task.

Simple test:

Then one CMD task before Publish Build Artifact task to choose specific folders:

xcopy "Source\Project\ABC Project\XYZ Packages" "$(build.artifactstagingdirectory)\XYZ Packages\" /s /e /h /y
xcopy "Tools\A" "$(build.artifactstagingdirectory)\A\" /s /e /h /y
xcopy "Tools\B" "$(build.artifactstagingdirectory)\B\" /s /e /h /y
xcopy "Tools\C" "$(build.artifactstagingdirectory)\C\" /s /e /h /y

Final content of the Drop artifact: