Escaping characters on AzureBlobContent

Question

I have got a problem to set my content in AzureBlobStorage. In local, I have succeeded to replace characters for each files in a directory.

$sourceFolder = "C:\MyDirectory"
$targetFolder = "C:\MyDirectoryEncodeded"
$fileList = Dir $sourceFolder -Filter *.dat
MkDir $targetFolder -ErrorAction Ignore
ForEach($file in $fileList) {   
    $file | Get-Content | %{$_ -replace '"',''} | %{$_ -replace ',','.'} | Set-Content -Path "tempDirectory\$file"
    $newFile = Get-Content "tempDirectory\$file"    
    $Utf8NoBomEncoding = New-Object System.Text.UTF8Encoding $False
    [System.IO.File]::WriteAllLines("targetDirectory\$file" , $newFile,$Utf8NoBomEncoding)
}
exit

But now, I need to do the same in Microsoft Azure. I get the content into an Azure Blob Storage, I escape characters, I encoding my file in UTF-8NoBom and then I set the encode file into a new Blob Directory.

Nevertheless, I faced an issue when I want to set the new content with escape characters (First line in my loop).

$storageContext = New-AzureStorageContext -ConnectionString "DefaultEndpointsProtocol=https;AccountName=<myAccountName>;AccountKey=<myAccountKey>;"
$sourceFolder = Get-AzureStorageBlob -Container "datablobnotencoded" -Blob "*.dat" -Context $storageContext
$targetFolder = Get-AzureStorageBlob -Container "datablob" -Context $storageContext 
MkDir $targetFolder -ErrorAction Ignore 
ForEach($file in $sourceFolder) {
    Get-AzureStorageBlob -Container "datablobnotencoded" -Blob $file.Name -Context $storageContext | Get-AzureStorageBlobContent | %{$_ -replace '"',''} | %{$_ -replace ',','.'} | Set-AzureStorageBlobContent -File $file.Name -Context $storageContext -CloudBlob $file
    $newFile = Get-AzureStorageFileContent -Path $file 
    $Utf8NoBomEncoding = New-Object System.Text.UTF8Encoding $False
    [System.IO.File]::WriteAllLines($file , $newFile, $Utf8NoBomEncoding)
}

I've got this error:

Set-AzureStorageBlobContent : Cannot bind parameter 'CloudBlob'. Cannot convert the "Microsoft.WindowsAzure.Commands.Storage.Model.ResourceModel.AzureStorageBlob" value of type "Microsoft.WindowsAzure.Commands.Storage.Model.ResourceModel.AzureStorageBlob" to type "Microsoft.WindowsAzure.Storage.Blob.CloudBlob". At line:7 char:264 + ... lobContent -File $file.Name -Context $storageContext -CloudBlob $file + ~~~~~ + CategoryInfo : InvalidArgument: (:) [Set-AzureStorageBlobContent], ParameterBindingException + FullyQualifiedErrorId : CannotConvertArgumentNoMessage,Microsoft.WindowsAzure.Commands.Storage.Blob.SetAzureBlobContentCommand

Thank you for your answers!

Is this ... -Context $storageContext $targetFolder = ... just a typo in the question? It should be on two separate lines or these lines should have been separated by a semicolon. — Theo
Are you getting valid folder paths in $sourceFolder and $targetFolder and should'n you be doing a Get-ChildItem (Dir) on the source folder first to iterate over the files in there? — Theo
Paths are correct because during debug I got the expected $file.Name. — Tristan Le Gall

Ivan Yang Ivan Yang · Accepted Answer · 2019-01-09T05:46:23

There are some mistakes in your powershell scripts:

1.You may misunderstand the usage of Get-AzureStorageBlobContent, it's used to download blob to local, you cann't get the content of the blob, more details refer here.

2.In the loop, you used $newFile = Get-AzureStorageFileContent -Path $file, the Get-AzureStorageFileContent cmdlet is for file share storage, not for the blob storage.

You can use Get-AzureStorageBlobContent to download the blobs to a local folder, then operate on the local file which is downloaded from blob storage. After the file is modified, you can use Set-AzureStorageBlobContent to upload the local files to the specified azure blob storage.

Sample code as below, and works fine at my side:

$context = New-AzureStorageContext -ConnectionString "xxxx"

#download the blobs in specified contianers
$sourceFolder_blob = Get-AzureStorageBlob -Container "test-1" -Blob "*.txt" -Context $context

#the target azure container, which you want to upload the modifed blob to
$taget_container="test-2"

#the local path which is used to store the download blobs, and make sure the folders exist before use.
$sourceFolder_local="d:\test\blob1\"
$targetFolder_local="d:\test\blob2\"

foreach($file in $sourceFolder_blob)
{

#download the specified blob to local path
Get-AzureStorageBlobContent -Container "test-1" -Blob $file.name -Destination $sourceFolder_local -Context $context

#get the local file path
$local_file_path=$sourceFolder_local + $file.name

#set content to the file in target local folder
$local_target_file_path = "$targetFolder_local"+$file.name


#since the files are downloaded to local, you can any operation for the local file
Get-Content $local_file_path | %{$_ -replace '-','!'} | %{$_ -replace ',','.'} | Set-Content -Path $local_target_file_path

$newFile = Get-Content -Path $local_target_file_path


$Utf8NoBomEncoding = New-Object System.Text.UTF8Encoding $False
[System.IO.File]::WriteAllLines($local_target_file_path , $newFile,$Utf8NoBomEncoding)

#the last step, upload the modified file to another azure container
Set-AzureStorageBlobContent -File $local_target_file_path -Context $context -Container $taget_container
}

Escaping characters on AzureBlobContent

1 Answers