convert Csv file into Xml on Azure data lake store using Azure Powershell runbook

5
votes

I want to convert CSV file into XML on Azure data lake store using Azure Powershell. I was using this code on Runbook of azure automation and it worked fine but No XML is being generated

$cred = Get-AutomationPSCredential -Name 'xyz'
$subscriptionName = 'Pay-As-You-Go'
$null = Add-AzureRmAccount -Credential $cred
Select-AzureRmSubscription -SubscriptionName $subscriptionName | Out-Null 
$adla = "TestAzure"
$obj = Get-AzureRmDataLakeStoreItem -AccountName $adla -Path "/custom/logile/masters/Test/Test.csv" 
 $xmlobj = ConvertTo-Xml -InputObject $obj -As 'string' 
$xmlobj | Out-File -FilePath "/custom/logile/masters/Test/xmlOut.xml"

This code is working fine till "$obj= Get" here, but I am not able to write into Azure data lake. Is there any way that I can use Object to write the file in Azure data lake.

1
powershellazure-data-factoryazure-data-lakeazure-runbook
Why can't Import-AzureRmDataLakeStoreItem be used to upload the file? What are the contents of $obj after the code runs? – AdminOfThings
@l--''''''---------'''''''''''' Do you have write access to /custom/logile/masters/Test/? – tukan

1 Answers

4
votes

Thanks for reaching out!

You were trying to export the xml file using Out-File cmdlet to an nonexistent path /custom/logile/masters/Test in current instance of the Azure Automation execution. And you were not uploading the xml file to a Data Lake Store.

To accomplish your requirement you would have to use the below commands by replacing last line in your runbook (i.e., last line is $xmlobj | Out-File -FilePath "/custom/logile/masters/Test/xmlOut.xml")

$xmlobj | Out-File -FilePath "$env:temp/xmlOut.xml"
Import-AzureRmDataLakeStoreItem -AccountName $adla -Path "$env:temp/xmlOut.xml" -Destination "/custom/logile/masters/Test/xmlOut.xml"

I have tested it successfully. If interested you may check below screenshots for illustration.

Shell power runbook

Test

Data Explorer

Note that the first few lines of my runbook in the above screenshot are a bit different from yours because I have given Automation Account RunAs account the permission to Data Lake store destination folder. So you may ignore this part.

Hope this helps!! Cheers!!