Copy from Azure Blob to AWS S3 using C#

Question

Please note that this is my first time doing anything in C# so please be kind, I might have made some very basic mistakes. (and yes I know I shouldnt hardcode keys but will fix it when the code does what I want).

I am trying to create an Azure Function that copies any new items from Blob storage to AWS S3. I have managed to copy from blob to blob using the code from this article: https://cmatskas.com/copy-azure-blob-data-between-storage-accounts-using-functions/

I have tried to amend that code to instead save to an AWS S3 bucket. While this code compiles successfully and gives me successful log entires, it doesn't copy any files. Any ideas?

using System;
using System.IO;
using Amazon.S3;
using Amazon.S3.Model;
using Amazon.S3.Transfer;
using Microsoft.Azure;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Blob;

public async static void Run(CloudBlockBlob myBlob, TraceWriter log) 
{
    await CopyBlob(myBlob, log);
}

private async static Task CopyBlob(CloudBlockBlob myBlob, TraceWriter log)
{
    var existingBucketName = "bucketname";
    var keyName = "backup";
    var accessKey = "key";
    var secretKey = "secretkey";

    TransferUtility fileTransferUtility = new TransferUtility(new AmazonS3Client(accessKey,secretKey,Amazon.RegionEndpoint.eu-west-2));

    log.Info("Starting Copy");

    try{
        using (var stream = await myBlob.OpenReadAsync())
        {
            await fileTransferUtility.UploadAsync(stream, existingBucketName, keyName);
        }
        log.Info("Copy completed");

    }
    catch(Exception ex){
        log.Error(ex.Message);
        log.Info("Copy failed");
    }
    finally{
        log.Info("Operation completed");
    }
}

Edit: Got it working for anyone finding this in the future.

using System;
using System.IO;
using Amazon.S3;
using Amazon.S3.Model;
using Amazon.S3.Transfer;
using Microsoft.Azure;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Blob;

public async static void Run(CloudBlockBlob myBlob, TraceWriter log) 
{
    await CopyBlob(myBlob, log);
}

private async static Task CopyBlob(CloudBlockBlob myBlob, TraceWriter log)
{
    var existingBucketName = "bucketname";
    var keyName = myBlob.Name;
    var accessKey = "accesskey";
    var secretKey = "secretkey";

    TransferUtility fileTransferUtility = new TransferUtility(new AmazonS3Client(accessKey,secretKey,Amazon.RegionEndpoint.eu-west-2));

    log.Info("Starting Copy");

    try{
        using (var stream = await myBlob.OpenReadAsync())
        {
            await fileTransferUtility.UploadAsync(stream,existingBucketName,keyName);
        }
        log.Info("Copy completed");

    }
    catch(Exception ex){
        log.Error(ex.Message);
        log.Info("Copy failed");
    }
    finally{
        log.Info("Operation completed");
    }
}

Yes, getting the following log output: 2017-10-21T19:07:39.907 Function started (Id=a4d8faee-1612-46cc- 8134-1c0b74ec4c2f) 2017-10-21T19:07:40.466 Starting Copy 2017-10-21T19:07:40.466 Function completed (Success, Id=a4d8faee- 1612-46cc-8134-1c0b74ec4c2f, Duration=565ms) 2017-10-21T19:07:41.123 Copy completed 2017-10-21T19:07:41.123 Operation completed — Hasnas

Fabio Cavalcante Fabio Cavalcante · Accepted Answer · 2017-10-23T17:44:19

You should be seeing a warning about this, but your void method is likely causing the issue here.

Please update your function code to the following:

public async static Task Run(CloudBlockBlob myBlob, TraceWriter log) 
{
    await CopyBlob(myBlob, log);
}

Notice the change from void to Task

Copy from Azure Blob to AWS S3 using C#

1 Answers