Azure DevOps: Copying files to az fileshare very slow

0
votes

When I mount an Azure Fileshare from my Azure DevOps pipeline and copy files to the file share via robocopy, this is with 80 Megabytes per minute (12 MBit/s) very slow:

               Total    Copied   Skipped  Mismatch    FAILED    Extras
    Dirs :       456       456       249         0         0         0
   Files :      4122      4122         0         0         0         0
   Bytes :   2.809 g   2.809 g         0         0         0         0
   Times :  18:36:49   0:35:04                       0:00:00   0:01:10
   Ended : Wednesday, January 29, 2020 11:52:31 PM

I use a StorageV2 account. Azure DevOps and the storage account are in different regions. This is the robocopy command I use:

robocopy ..\my_directory z:\my_directory /E /NS /NC /NP /XD .git /MT:32 /W:5

... and this is how I have mounted the azure fileshare inside the pipeline:

New-PSDrive -Name $(storage.drive) -PSProvider FileSystem -Root "$(storage.unc)" -Persist

Is this normal? How can I make it significantly faster?

2
azureazure-devopsazure-storageazure-pipelines
Hi friend, have you tried to run that robocopy command locally? Is it also slow like using Devops pipeline ? – LoLance
Good question! - I have tried it now, and it took one minute and one second to copy the same amount of files "locally" within the Microsoft Azure Cloud Agent (instead of 35:04 minutes). – hey

2 Answers

1
votes

It got lightning fast (1 minute, 30 seconds) when I moved the azure fileshare to the same region as the Azure DevOps Pipeline.

The region of Azure DevOps is displayed in the organizational settings of Azure Devops.

1
votes

For this scenario, instead of using robocopy we recommend using AzCopy v10.4 or later. Unlike robocopy, it's optimized for transferring over long distances to cloud storage.

(We also have various fuller-featured data transfer tools, but for a user experience directly comparable to robocopy, but with faster speed, AzCopy is the way to go).