I have a Azure Data Lake Store gen1 (ADLS-1) and a Azure Data Factory (ADF) (V2) with Data Flow (DF). When I create a new DF in ADF and select in the Source and/or Sink node a dataset from ADLS-1, I get the following validation` error (in DF):
source1
AzureDataLakeStore does not support MSI authentication in Data Flow.
Does this mean that I cannot use DF with ADLS-1 or is this some kind of authentication problem?
List of thing I've tried:
- I have given the ADF resource an Owner role in Access control (IAM) of the ADLS-1
- I have given the ADF resource all (read, write, etc) permissions in the ADLS-1 folder of the dataset
- I can copy data from and to the ADLS-1 in a ADF pipeline (so outside DF)
- I can select datasets in the source and sink node of DF for datasets from ADLS-2 (gen 2) (so here I didn't get the error)
- I can create a pipeline which copies first a dataset from ADLS-1 to ADLS-2 and then process it with DF (and copy it back). This workaround is pretty tedious and I do not have a ADLS-2 in production (for now).
- It says here that the supported capabilities for ADLS-1 includes Mapping data flow (DF).
If someone knows a method to use DF with ADLS-1 or rule out its capabilities that would be pretty helpful.
