Copy Azure Blob With Snapshots

Question

I'm attempting to backup some blob storage to prevent against accidental deletion of a container, and I can copy the blobs fine either using PowerShell and start-CopyAzurBlog, or using AZCopy from the command line, however these blobs have a number of snapshots (used as part of a versioning process) and whenever I copy, it does not take the snapshots with it.

I am aware that you can use the /snapshot command with AZCopy, but this creates a whole new blob for each snapshot, I need them to part of the single blob. Is there anyway to copy these intact? Ideally this would be in PowerShell so I can use Azure automation, but happy to do something in C# if needs be.

Answer 1

I have to point out that the feature you're asking is very difficult to implement.

Snapshot Blob API only supports creating a snapshot for the current blob data. Therefore, in order to backup snapshots, we need to create the blob with the same data as first snapshot in destination, then create a snapshot; and overwrite the target blob with the same data as the second snapshot, then create a snapshot; ... ; continue doing that until all the snapshots are created in destination, at last overwrite the target blob with the same data as current source blob.

Since the capacity at source blob are actually not billed as sum of all snapshots and current blob, if you'd like to save the price on destination as well after backup, you have to diff between two snapshots or between the last snapshot & current blob, and only overwrite the different blocks/pages at each step, which may introduce much more complexity.

Answer 2

Here's the code to go with Zhaoxing Lu's logic. The approach is to first list the snapshots of the source blob, create blob in the destination from the snapshots and take snapshots of that blob. Finally you would copy the base blob into target blob.

    static void CopyBlobAndSnapshots()
    {
        var sourceAccountName = "<source-account-name>";
        var sourceAccountKey = "<source-account-key>";
        var sourceContainerName = "<source-container-name>";
        var targetAccountName = "<target-account-name>";
        var targetAccountKey = "<target-account-key>";
        var targetContainerName = "<target-container-name>";
        var blobName = "<source-blob-name>";
        var sourceAccount = new CloudStorageAccount(new StorageCredentials(sourceAccountName, sourceAccountKey), true);
        var targetAccount = new CloudStorageAccount(new StorageCredentials(targetAccountName, targetAccountKey), true);
        var sourceBlobClient = sourceAccount.CreateCloudBlobClient();
        var sourceContainer = sourceBlobClient.GetContainerReference(sourceContainerName);
        var sourceBlob = sourceContainer.GetBlockBlobReference(blobName);
        var targetBlobClient = targetAccount.CreateCloudBlobClient();
        var targetContainer = targetBlobClient.GetContainerReference(targetContainerName);
        targetContainer.CreateIfNotExists();
        //Create a SAS Token on source blob with read permissions that is valid for 2 weeks.
        var sasToken = sourceBlob.GetSharedAccessSignature(new SharedAccessBlobPolicy()
            {
                Permissions = SharedAccessBlobPermissions.Read,
                SharedAccessExpiryTime = new DateTimeOffset(DateTime.UtcNow.AddDays(14))
            });
        //List blob snapshots first
        var baseBlobAndSnapshots = sourceContainer.ListBlobs(blobName, true, BlobListingDetails.Snapshots).ToList();
        //Since the list contains both base blob and snapshots, we should remove the base blob from this list.
        var blobSnapshots = baseBlobAndSnapshots.Where(b => ((CloudBlockBlob)b).SnapshotTime != null).ToList();
        //Now we should arrange them in reverse chronological order
        blobSnapshots.Reverse();
        CloudBlockBlob targetBlob = null;
        string sourceBlobUrl = string.Empty;
        foreach (var blob in blobSnapshots)
        {
            var blockBlob = (CloudBlockBlob) blob;
            Console.WriteLine("Copying blob snapshot. Snapshot date/time = " + blockBlob.SnapshotTime);
            sourceBlobUrl = string.Format("{0}&{1}", blockBlob.SnapshotQualifiedUri, sasToken.Substring(1));
            targetBlob = targetContainer.GetBlockBlobReference(blobName);
            targetBlob.StartCopy(new Uri(sourceBlobUrl));
            //Check the status;
            targetBlob.FetchAttributes();
            do
            {
                var copyState = targetBlob.CopyState;
                if (copyState.Status == CopyStatus.Pending)
                {
                    Thread.Sleep(1000);//Copy not completed. Wait for it to complete.
                    targetBlob.FetchAttributes();
                }
                else
                {
                    break;
                }
            } 
            while (true);
            Console.WriteLine("Copying blob snapshot complete. Snapshot date/time = " + blockBlob.SnapshotTime);
            Console.WriteLine("----------------------------");
            //Now take the blob snapshot
            Console.WriteLine("Taking blob snapshot....");
            targetBlob.Snapshot();
            Console.WriteLine("Blob snapshot taken....");
            Console.WriteLine("----------------------------");
        }
        Console.WriteLine("Copying base blob.");
        sourceBlobUrl = string.Format("{0}{1}", sourceBlob.Uri, sasToken);
        targetBlob = targetContainer.GetBlockBlobReference(blobName);
        targetBlob.StartCopy(new Uri(sourceBlobUrl));
        //Check the status;
        targetBlob.FetchAttributes();
        do
        {
            var copyState = targetBlob.CopyState;
            if (copyState.Status == CopyStatus.Pending)
            {
                Thread.Sleep(1000);//Copy not completed. Wait for it to complete.
                targetBlob.FetchAttributes();
            }
            else
            {
                break;
            }
        } 
        while (true);
        Console.WriteLine("Blob with snapshot copy completed!");
    }