Copy blob without downloading it to local memory

2
votes

I need to copy blob from one Azure storage account to another. As mentioned here,

AzCopy does it by downloading the blobs from the source storage endpoint to local memory and then upload them to the destination storage endpoint.

Due to some constraints I need a solution which can directly copy blob from one Azure storage account to another without downloading blob to local memory. Also I need to do a synchronous copy.

Is there any tool to achieve this or any other solution approach feasible enough to build on our own?

enter image description here

1
azureazure-storage-blobsazcopy
Just to clarify: AzCopy, by default, does not download content locally; the transfer happens within Azure itself. It's only when you choose 'synchronous copy' that it has to download and re-upload, as AzCopy controls every bit of the transfer. Curious why you need this to be a synchronous operation? Is there an issue with the current async copy? – David Makogon
@DavidMakogon: In async copy, copying of data will run in the background using spare bandwidth capacity that has no SLA in terms of how fast a blob will be copied. I need consistent speed which sync copy provides. But due to some constraints I cannot afford sync copy provided by AzCopy as it downloads the blob to local memory and then uploads it. Instead of a two step process, I wish it to be one step. – Shridhar R Kulkarni
Honestly, you should benchmark. I think you'll find that the speeds within Azure are going to be faster than downloading+uploading, even if you download/upload via a VM in Azure. True, you won't get consistent speed, but you should consider whether, on average, it's "fast enough." – David Makogon
Sure. Thanks for the response :) – Shridhar R Kulkarni

1 Answers

1
votes

Due to some constraints I need a solution which can directly copy blob from one Azure storage account to another without downloading blob to local memory. Also I need to do a synchronous copy.

I don't think it is possible. If you want synchronous copy then the only option is to download the blob and then reupload it. To reduce the time for synchronous copy, you can run AzCopy from a VM in Azure itself.

If you want to directly copy from one storage account to another then the only option available to you is perform asynchronous copy which is available in AzCopy.