The issue is that I am unable to write to an Azure Data Lake Store in one Azure subscription from an Azure Data Factory in another subscription without enabling read/write/execute access for ALL USERS AND GROUPS in the Data Lake Store. I know that I set up the authorization properly in the ADF Linked Service for the Data Lake Store too, so that is not the issue. Here are the details:
I have an Azure Data Lake Store (ADLS) created by me under my company's main Azure subscription (in this sub I am a Co-admin and have access to everything except creating/modifying other users).
I have an Azure Data Factory (ADF) under my own MSDN-related Azure subscription and I am trying to use the ADLS as the sink for a copy activity.
I use the same login to access both Azure subscriptions in to portal, and when I run my pipeline I get a (403) Forbidden error with ErrorCode=UserErrorAdlsUnauthorizedAccess.
What I have tried so far:
I followed the instructions here https://azure.microsoft.com/en-us/documentation/articles/data-lake-store-secure-data/ and added myself to the AAD group. However when I went to add the group as an owner of the data store, Azure could not find the group so I was unable to add it.
AAD Group:
Failure to find AAD Group:
I also tried adding myself as a user in the OWNER role on the data store (via the Users blade) and I have read/write/execute access in the Access blade of the data lake store. There was also another developer added as an Owner, and had read/write/execute access but the same results were produced.
Me assigned as a user to data lake store:
Me granted access to file system of data lake store:
Both of these result in the same 403 unauthorized error when I try to run the ADF pipeline or re-run any of the failed slices. The only thing that has worked for me is to enable READ/WRITE/EXECUTE access for all users and groups:
I tried with write/execute and with just write, but neither of those combinations worked. When I enable that and re-run my failed slice/s now, everything completes successfully and data is written to the Data Lake Store.
Is there any way to write to a data lake store from another subscription without opening up access like this? I would think so due to the fact that you can specify a subscription in the Data Lake Store Linked Service, but so far I haven't been able to do it...
