Error writing a file to Azure Data Lake from an Azure function using MSI

3
votes

I am trying to create an Azure function that writes to Azure Data Lake Store. I am using Managed Service Identity to, well, manage the authentication stuff.

I have enabled MSI on the Function app. I have also enabled the Function app to access the required Data Lake Store. And I am using the following code to get a token and write to ADL. AM I missing something?

var azureServiceTokenProvider = new AzureServiceTokenProvider();
string accessToken = await azureServiceTokenProvider.GetAccessTokenAsync("https://datalake.azure.net");
var client = AdlsClient.CreateClient(_adlsAccountName, accessToken);
using (var stream = client.CreateFile(fileName, IfExists.Overwrite))
    {
        byte[] textByteArray = Encoding.UTF8.GetBytes("Winter is coming! \r\n");
        stream.Write(textByteArray, 0, textByteArray.Length);
    }

My code fails with the following error.

with exception Microsoft.Azure.DataLake.Store.AdlsException: Error in creating file /Path/tempFile0.txt.

**Operation: CREATE failed with HttpStatus:Unauthorized Error: Uexpected error in JSON parsing.**

Last encountered exception thrown after 1 tries. [Uexpected error in JSON parsing]

[ServerRequestId:<Some ID>]

at Microsoft.Azure.DataLake.Store.AdlsClient.CreateFile(String filename, IfExists mode, String octalPermission, Boolean createParent)
2
azure-functionsazure-data-lakeazure-managed-identity

2 Answers

4
votes

Adding "Bearer " to the access token worked for me. Like thus (everything else remains the same),

var client = AdlsClient.CreateClient(_adlsAccountName, “Bearer “ + accessToken);

Thanks in part to Arturo Lucatero's Github doc which mentioned this. https://github.com/MicrosoftDocs/azure-docs/blob/master/articles/active-directory/managed-service-identity/tutorial-linux-vm-access-datalake.md

0
votes

I mainly use following snippet to authenticate and write to Data Lake from Azure Functions:

var clientCredential = new ClientCredential(clientId, clientSecret);
var creds = ApplicationTokenProvider.LoginSilentAsync("domainId", clientCredential).Result;
_client = new DataLakeStoreFileSystemManagementClient(creds);

Where clientId and clientSecret are:

  • an ObjectId from AD
  • a secret attached to it

So basically you have to create a service principal and obtain those particular properties from the portal.

Then I'm able to use following:

public async Task AppendToFile(string destinationPath, string content)
{
    using (var stream = new MemoryStream(Encoding.UTF8.GetBytes(content)))
    {
        await _client.FileSystem.ConcurrentAppendAsync("datalakeaccount", destinationPath, stream, appendMode: AppendModeType.Autocreate);
    }
}

To write data to ADLS.

You can also refer to this blog post.