1
votes

Is there a way to do this?

I have plenty of files over few servers and Amazon s3 storage and need to upload to Azure from an app (Java / Ruby)

I prefer not to download these files on my app server and then upload it to Azure blob storage.

I've checked the Java and Ruby sdk, it seems there's no straight way to do this based on the examples (means I have to download these files first on my app server and upload it to Azure)

Update: Just found out about CloudBlockBlob.startCopy() in the Java SDK. Tried it and it's basically what I want without using Third party tools like AzCopy.

2
I wrote a blog post long ago on copying a S3 bucket in Azure Blob Container that you may find useful: gauravmantri.com/2012/06/15/…. Please note that some of the functions used there may be deprecated but this post should give you an idea about how to implement this functionality.Gaurav Mantri
Hi @GauravMantri, thanks for the post and I did search on google and found your blog post before opening this question since I was looking for solution in Java. You're right the functionality has pretty much been deprecated but it did show me where to look for. Thank you again for commenting and the great blog post.edthix
FYI: AzCopy is not a third party tool. It is Microsoft created and maintained.Emily Gerner

2 Answers

1
votes

You have a few options, mostly licensed, but I think that AzureCopy is your best free alternative. You can check a step by step experience on MSDN Blogs.

All you need is your Access Keys for both services and with a simple command:

azurecopy -i https://mybucket.s3-us-west-2.amazonaws.com/ -o https://mystorage.blob.core.windows.net/mycontainer -azurekey %AzureAccountKey% -s3k %AWSAccessKeyID% -s3sk %AWSSecretAccessKeyID% -blobcopy -destblobtype block

You can pass blobs from one container to the other.

1
votes

As @EmilyGerner said, AzCopy is the Microsoft offical tool, and AzureCopy that @MatiasQuaranta said is the third party tool on GitHub https://github.com/kpfaulkner/azurecopy.

The simple way is using AWS Command Line and AzCopy to copy all files from S3 to local directory to Azure Blob Storage. You can refer to my answer for the other thread Migrating from Amazon S3 to Azure Storage (Django web app). But it is only suitable for a small data size bucket.

The other effective way is programming with SDKs of Amazon S3 and Azure Blob Storage for Java. Per my experience, Azure SDK APIs for Java is similiar with C#‘s, so you can try to refer to Azure Blob Storage Getstarted doc for Java and AWS SDK for Java to follow @GauravMantri sample code to rewrite code in Java.