Azure Function can write files in Data Lake, when it is bound to Event Grid, but not when it is called from Azure Data Factory

0
votes

I have an Azure Function that should process zip files and convert their contents to csv files and save them on a Data Lake gen 1.

I have enabled managed Identity of this Azure Functions. Then I have added this managed Identity as an OWNER on the Access control (IAM) of Data Lake.

First Scenario:

I call this Azure function from Azure Data Factory, and send the fileuri of zip files, which are persisted in a Storage Acccount from Azure Data Factory to Azure Functions, Azure Function process the files by saving csv files in data lake I get this error:

Error in creating file root\folder1\folder2\folder3\folder4\test.csv.

Operation: CREATE failed with HttpStatus:Unauthorized Token Length: 1162

Unknown Error: Unexpected type of exception in JSON error output. Expected: RemoteException Actual: error Source: Microsoft.Azure.DataLake.Store StackTrace: at Microsoft.Azure.DataLake.Store.WebTransport.ParseRemoteError(Byte[] errorBytes, Int32 errorBytesLength, OperationResponse resp, String contentType).

RemoteJsonErrorResponse: Content-Type of error response: application/json; charset=utf-8.

Error:{"error":{"code":"AuthenticationFailed","message":"The access token in the 'Authorization' header is expired.

Second Scenario

I have set an Event Grid for this Azure Functions. After dropping zip files in storage account, which is bound to Azure Functions, zip files are processed and csv files are successfully saved in the Data Lake.

good to know azure function and Data Lake are in the same vnet

Could someone explain to me, why my function works fine with the event grid but doesn't work if I call it from Azure Data Factory? (saving csv files in Data Lake Gen 1)

1
azureazure-functionsazure-data-factoryazure-data-lakeazure-managed-identity
which kind of azure function did you use? – Cindy Pau

1 Answers

0
votes

Have you show all of the error?

From the error seems it is related to Azure Active AD authentication(Data Lake Storage Gen1 uses Azure Active Directory for authentication). Have you use Bearer token in your Azure Function when you try to send something to Data Lake Storage Gen1?

please show the code of your azure function, otherwise it will be hard to find the cause of the error.