Copy Blobs from one storage account to another storage account using Python

Question

I'm trying the following code to copy all blobs from one storage account into another, in the same resource group:

src_storage_client = BlobServiceClient.from_connection_string(src_storage['connectionString'])
src_container = src_storage_client.get_container_client(src_storage['containerName'])

dst_storage_client = BlobServiceClient.from_connection_string(dst_storage['connectionString'])
dst_container = dst_storage_client.get_container_client(dst_storage['containerName'])

try:
    for blob in src_container.list_blobs():            
        src_blob = BlobClient.from_connection_string(src_storage['connectionString'], src_storage['containerName'], blob.name)
        new_blob = BlobClient.from_connection_string(dst_storage['connectionString'], dst_storage['containerName'], blob.name)
        new_blob.start_copy_from_url(src_blob.url)

I receive the following error:

azure.core.exceptions.ClientAuthenticationError: Operation returned an invalid status 'Server failed to authenticate the request. Please refer to the information in the www-authenticate header.'

I tried generate_blob_sas and generate_account_sas to no use! I checked out other examples, documentations, and repos but none matched my case, unless I missed something.

Using that same new_blob instance with upload method, it works perfectly. However, I want to copy rather than download-then-upload-again.

Pamela Peng Pamela Peng · Accepted Answer · 2021-06-08T14:36:23

The code in your issue is correct. And you could use container.get_blob_client(blobName) instead in the loop.

for blob in src_container.list_blobs():            
    src_blob = src_container.get_blob_client(blob.name)
    new_blob = dst_container.get_blob_client(blob.name)
    new_blob.start_copy_from_url(src_blob.url)

It's important to check if src_blob.url or blob.name contain the special characters or space, they need to be encode first. It seems caused by special blobs, please try with a blob. When you use SAS token, you should check the allowed permissions and expired time.

Copy Blobs from one storage account to another storage account using Python

1 Answers