How to append data into existing Azure data lake file?

2
votes

I am trying to append file in existing data lake storage file. Using azure CLI command I successfully append the data into file.But appended data append into single line only.Because of this file structured become changed. for this I run following command using Azure CLI.

First I read test1.txt from azure data lake and store output into one variable.

$data = az dls fs preview --account xxxxadls --path /Input/xxx/dbo.test_1.txt  --output table

After this I read line by line records and append into existing test file on data lake. 

foreach($ds in $data) {
   az dls fs append --account xxxxadls --path /Input/xxx/dbo.test.txt --content $ds
}

finally when I check data in data lake then all data append in single line and it shows like below. enter image description here

So, Anyone having any idea like how can I break the line while I append the data in data lake.

1
azureazure-data-lakeazure-cli

1 Answers

1
votes

It seems to be caused by the $ds value in the code foreach($ds in $data) that be missing new line symbol.

According to the offical blog PowerTip: New Lines with PowerShell, you can try to manually add the new line symbol for powershell to the end of $ds as the code below to fix it.

foreach($ds in $data) {
   az dls fs append --account xxxxadls --path /Input/xxx/dbo.test.txt --content "$($ds)`n"
}