Azure Datalake operation returned an invalid status code forbidden

2
votes

I am writing a simple file to Azure Datalake to learn how to use this for other means, but I ma having issues and when I try to write I get the following error message

21/5/2018 9:03:27 AM] Executed 'NWPimFeederFromAws' (Failed, Id=39adba4b-9c27-4078-b560-c25532e8432e)
[21/5/2018 9:03:27 AM] System.Private.CoreLib: Exception while executing function: NWPimFeederFromAws. Microsoft.Azure.Management.DataLake.Store: One or more errors occurred. (Operation returned an invalid status code 'Forbidden'). Microsoft.Azure.Management.DataLake.Store: Operation returned an invalid status code 'Forbidden'.

The code in question is as follows

static void WriteToAzureDataLake() {

    // 1. Set Synchronization Context
    SynchronizationContext.SetSynchronizationContext(new SynchronizationContext());

    // 2. Create credentials to authenticate requests as an Active Directory application
    var clientCredential = new ClientCredential(clientId, clientSecret);

    //var creds = ApplicationTokenProvider.LoginSilentAsync(tenantId, clientCredential).Result;
    var creds = ApplicationTokenProvider.LoginSilentAsync(tenantId, clientCredential).Result;

    // 2. Initialise Data Lake Store File System Client
    adlsFileSystemClient = new DataLakeStoreFileSystemManagementClient(creds);

    // 3. Upload a file to the Data Lake Store
    var source = "c:\\nwsys\\source.txt";
    var destination = "/PIMRAW/destination.txt";
    adlsFileSystemClient.FileSystem.UploadFile(adlsAccountName, source, destination, 1, false, true);

    // FINISHED
    Console.WriteLine("6. Finished!");
}

I have added the application from my Azure AD to the access list on that specific folder I am trying to write to as follows

Azure portal showing the correct rights

The clientID and clientSecret in my code comes from this app so I am a bit lost as to why I get forbidden.

Have I forgotten anything else?

Could it be that the loginAsync has not yet finished before I try and create the client?

3
c#azureazure-active-directoryazure-functionsazure-data-lake
did you replace child permissions for the folder with permissions on folder? also how long did you wait after applying those. it usually takes like 30 minutes for permissions to propagete - 4c74356b41
I did say this folder and all children yes and I have waited more than 30 minutes - Matt Douhan
It's not related to LoginAsync (I can use your same code without issues). Is there any chance the file already exists, and you haven't granted access to the specific file (in order to over-write it)? - Philippe Signoret
The file does not exist for sure the folder is new and totally empty - Matt Douhan

3 Answers

1
votes

Did you give your application/service principal execute access to the parent folders in the path to the specific folder to which you're app is writing? This is needed to travers the folder path, see here for some examples: https://docs.microsoft.com/en-us/azure/data-lake-store/data-lake-store-access-control#common-scenarios-related-to-permissions.

0
votes

Could it be that the loginAsync has not yet finished before I try and create the client?

It is not related to loginAsync. Based on my test, it works correctly on my side if I assign the permissions to the folder.

enter image description here

Test Result:

enter image description here

If it is possible, you could create a new Datalake account or new folder and try it again. I recommand that you could use fiddler to capture the detail information about exception.

0
votes

Not an answer, just documenting what I found when faced with a similiar error.

I added my Azure Data Factory Managed Identity to the contributor role at the account level (and therefore file system) level.

When trying to create blobs from ADF I got a forbidden error

So I added it to Storage Blob Data Contributor. It didn't work immediately but took about 10 minutes to be recognised. Then everything worked.