0
votes

We are using a Python Azure Function for ETL based on several files that are dropped off in blob storage every day at the same time. Our current workflow is using several Python functions to pick up those files (using the azure-storage-blob Python library), transform them, load them to our Azure SQL database, and then archive all the files to cold storage. Right now we are relying on timer triggers because some of the functions depend on other functions to be complete before manipulating and archiving the source files.

It seems like using Azure Durable Functions would be a better workflow for this, since we could orchestrate the functions better and decide when either the next ETL process should run or when the files should be archived. The problem is that Azure Durable Functions is not yet supported in Python. Is it possible to use a C# Durable Function to orchestrate a Python Azure Function? Or does Microsoft recommend using the preview Python Durable Functions (https://github.com/Azure/azure-functions-durable-python)? The documentation says that Python Durable Functions should have consumption plan support by May 2020.

1
you can call the python functions via http webhook if that trigger alone sufficesAravind

1 Answers

0
votes

Also, have you considered using Logic Apps as the orchestrator for your functions? Seems like that would be a more rapid way to implement orchestration.