Azure automation, PowerShell to fetch a file in private blob container

6
votes

I have an azure blob container set to private. I want to download files in this container using PowerShell.

This is what I put, however it is giving me ResourceNotFound error every time. Even when I put -Credential and my user name/access key. When I switch the container to public access, it always works. So am I missing anything?

Invoke-WebRequest -Uri $uri -OutFile $filePath
3
powershellazure

3 Answers

5
votes

Using Invoke-WebRequest is analogous to opening a link in your browser. It's a legitimate way to download files from Azure Storage, however to do that you'll need the URI to include a SAS (Shared Access Signature), which you'll have to have generated before you use it in your code. The PowerShell to achieve this is:

#Download via URI using SAS
$BlobUri = 'https://yourstorageaccount.blob.core.windows.net/yourcontainer/yourfile.txt'
$Sas = '?sv=2015-04-05&st=2015-04-29T22%3A18%3A26Z&se=2015-04-30T02%3A23%3A26Z&sr=b&sp=rw&sip=168.1.5.60-168.1.5.70&spr=https&sig=Z%2FRHIX5Xcg0Mq2rqI3OlWTjEg2tYkboXr1P9ZUXDtkk%3D'
$OutputPath = 'C:\Temp\yourfile.txt'
$FullUri = "$BlobUri$Sas"
(New-Object System.Net.WebClient).DownloadFile($FullUri, $OutputPath)

Alternatively, if you have the Azure PowerShell module installed, you can do it without any of that added pain:

# Download via Azure PowerShell
$StorageAccountName = 'yourstorageaccount'
$StorageAccountKey = Get-AzureStorageKey -StorageAccountName $StorageAccountName
$StorageContext = New-AzureStorageContext $StorageAccountName -StorageAccountKey $StorageAccountKey.Primary
$FileName = 'yourfile.txt'
$OutputPath = 'C:\Temp'
$ContainerName  = 'yourcontainer'
Get-AzureStorageBlobContent -Blob $FilebName -Container $ContainerName -Destination $OutputPath -Context $StorageContext
2
votes

I ended up to resolve similar requirement with Azure PowerShell Az module as follows:

$BlobFilePath = 'dir\blob.file' # Relative path in blob starting from container
$OutputFilePath = 'C:\temp\blob.file' # Path to download the file to
$StorageAccountName = 'storageaccountname'
$ContainerName = 'blob-container-name'

# Prompt for Azure Account creds, if working from VM with managed identity could add also switch -Identity to use that identity directly
Connect-AzAccount
$StorageContext = New-AzStorageContext -StorageAccountName $StorageAccountName

Get-AzStorageBlobContent -Blob $BlobFilePath -Container $ContainerName -Destination $OutputFilePath -Context $StorageContext
1
votes
$StartTime = $(get-date)
$datetime = $(get-date -f yyyy-MM-dd_hh.mm.ss)

$connection_string = ''
$AzureBlobContainerName = ''
 
$destination_path = "c:\download"


If(!(test-path $destination_path))
{
    New-Item -ItemType Directory -Force -Path $destination_path
}
$storage_account = New-AzStorageContext -ConnectionString $connection_string
 
# Download from all containers
#$containers = Get-AzStorageContainer -Context $storage_account
 
# Download from specific container
$containers = Get-AzStorageContainer -Context $storage_account | Where-Object {$_.Name -eq "$AzureBlobContainerName"}
 
$containers
Write-Host 'Starting Storage Dump...'
foreach ($container in $containers)
{
    Write-Host -NoNewline 'Processing: ' . $container.Name . '...'
    
    $blobs = Get-AzStorageBlob -Container $container.Name -Context $storage_account
    
    $container_path = $destination_path + '\' + $container.Name 
    new-item -ItemType "directory" -Path $container_path
    Write-Host -NoNewline ' Downloading files...'   
    foreach ($blob in $blobs)
    {       
        $fileNameCheck = $container_path + '\' + $blob.Name      
        if(!(Test-Path $fileNameCheck ))
        {
            Get-AzStorageBlobContent -Container $container.Name -Blob $blob.Name -Destination $container_path -Context $storage_account
        }           
    } 
    Write-Host ' Done.'
}
Write-Host 'Download complete.'

$elapsedTime = $(get-date) - $StartTime

$totalTime = "{0:HH:mm:ss}" -f ([datetime]$elapsedTime.Ticks)

Write-Output " -OK $totalTime" | Out-String