Azure Powershell Function App Reading Content From Data Lake Gen 1

Question

I would be very grateful for any assistance with this problem.

All I am trying to do is use a simple Azure function app using Powershell to read a file ("/input/specimen.json") in from my data lake, work with it, then eventually write it back out to the lake. I'm stuck already.

Here is the relevant code from the function app:

# The below line is at the very top of the function app run.ps1
using namespace System.Management.Automation.PSCredential

# The lines below are in the main area of the function app which are called.
$path = "/input/specimen.json"
$account = "mydatalake"

$tenant = "<my-tenant-id>"
$user = "<application/client id>"
$pass = "<client secret>"

$password = ConvertTo-SecureString $pass -AsPlainText -Force
$cred = New-Object -TypeName System.Management.Automation.PSCredential -argumentlist $user,$password

connect-azaccount -credential $cred -tenant $tenant -ServicePrincipal

$data = Get-AzDataLakeStoreItemContent -accountname $account -path $path
write-host $data

$body = $data | out-string
disconnect-azaccount

The output from the function app is:

2020-06-04T19:49:09Z [Error] ERROR: Get-AzDataLakeStoreItemContent : Error opening a Read Stream for file /stan/interests.json. Operation: GETFILESTATUS failed with HttpStatus:Forbidden RemoteException: AccessControlException GETFILESTATUS failed with error 0x83090aa2 (Forbidden. ACL verification failed. Either the resource does not exist or the user is not authorized to perform the requested operation.). [a6cfab98-149a-46eb-8cb6-bd0a40a11796] failed with error 0x83090aa2 (Forbidden. ACL verification failed. Either the resource does not exist or the user is not authorized to perform the requested operation.). [a6cfab98-149a-46eb-8cb6-bd0a40a11796][2020-06-04T12:49:08.1314705-07:00] JavaClassName: org.apache.hadoop.security.AccessControlException.

The error reeks of ACLS and permissions, but I have set up a Service Principal to use for the 'connect-azaccount' and have made that SP both a Contributor and Owner of the data lake, but still get the same errors. I'm reasonably certain that the SP I am using IS being used to authenticate because the Function App logs show the created session being echoed out to the host with the correct tenant and client id.

This is beginning to drive me crazy. What else do I need to get this to work? Could there be other roles or permissions needed? Is there a better way to do this?

Any and all assistance would be greatly appreciated - thanks in advance.

This may not necessarily resolve your specific issue, but consider using a managed identity instead of hardcoding secrets in your function. If you enable a system-managed identity (docs.microsoft.com/en-us/azure/app-service/…) and give it appropriate permissions, the Connect-AzAccount -Identity call in the auto-generated profile.ps1 will perform the authentication automatically. — Anatoli Beliaev

Jim Xu Jim Xu · Accepted Answer · 2020-06-05T08:09:19

If you want to use the service principal to access Azure data lake gen1 file, we need to configure ACL fro the sp. For more details, please refer to the document.

The ACL for Azure data lake gen1 has three permissions. There are Read, Write, and Execute. For more details, please refer to the docuemnt.

For example, we want to read file /Oregon/Portland/Data.txt, we need to configure ACL as below

Regarding how to configure it, we can do that on Azure Portal

Azure Powershell Function App Reading Content From Data Lake Gen 1

1 Answers