Azure Data Factory and SharePoint

Question

I have some Excel files stored in SharePoint online. I want copy files stored in SharePoint folders to Azure Blob storage.

To achieve this, I am creating a new pipeline in Azure Data factory using Azure Portal. What are possible ways to copy files from SharePoint to Azure blob store using Azure Data Factory pipelines?

I have looked at all linked services types in Azure data factory pipeline but couldn't find any suitable type to connect to SharePoint.

Answer 1

Rather than directly accessing the file in SharePoint from Data Factory, you might have to use an intermediate technology and have Data Factory call that. You have a few of options:

1. Use a Logic App to move the file
2. Use an Azure Function
3. Use a custom activity and write your own C# to copy the file.

To call a Logic App from ADF, you use a web activity. You can directly call an Azure Function now.

Answer 2

We can create a linked service of type 'File system' by providing the directory URL as 'Host' value. To authenticate the user, provide username and password/AKV details. Note: Use Self-hosted IR

Answer 3

You can use the logic app to fetch data from Sharepoint and load it to azure blob storage and now you can use azure data factory to fetch data from blob even we can set an event trigger so that if any file comes into blob container the azure pipeline will automatically trigger.