Copy files from AWS S3 to Azure storage archive tier

0
votes

I'm trying to copy files from AWS S3 to Azure storage archive access tier directly. Using azcopy I can copy the files from S3 to Azure but when using --block-blob-tier Archive flag I hit the error:

failed to parse user input due to error: blob-tier is not supported while copying from sevice to service

The only way around this I can find is to either download the files locally and then upload to Azure or change the access tier once uploaded. The first is the better solution as with the second it would be twice the cost (pay for the write operation when changing tier). Does anyone have another option or know why azcopy doesn't support this?

azcopy error: https://github.com/Azure/azure-storage-azcopy/blob/7eb7b49f1354b7c205dd469c1ac18cb12f7d353a/cmd/copy.go#L557

2
amazon-s3azure-storageazcopy

2 Answers

1
votes

Instead of downloading files from S3 to your local computer and then uploading it back to Azure Storage, you can simply copy the file from S3 to Azure Storage without explicitly setting the blob access tier. The resulting blob's access tier would be Hot. Once the copy operation completes successfully, you can change the access tier.

The advantage of this approach is that the copying happens directly between S3 and Azure Storage. However this process is asynchronous and you must wait for the copy operation to complete (instead of just getting it accepted) before initiating the access tier change operation.

UPDATE

If copying directly from S3 to Azure Blob Storage and then changing access tier is not practical for you from cost perspective, you can download the object from S3 and upload directly in Azure Blob Storage Archive tier. This feature is in preview as of writing of this post. This feature is available in Storage REST API version 2019-02-02. From the release notes link:

The Copy Blob, Put Block List, and Put Blob APIs support the x-ms-access-tier header for Block Blobs, to set the tier on the result without needing a second API call.

0
votes

you can use Flexify.io portal in which you need to add the s3 bucket credentials and azure blob storage and key then copy from s3 to block and vice versa is allowed. even move of data is