How to wait for Azure Data Factory pipeline to complete in Azure DevOps?

Question

In my Azure DevOps release, I need to trigger an Azure Data Factory pipeline and wait for the process to finish.

Is there any way to do this without any special trick in Az DevOps? Currently using vsts-publish-adf in my release.

Thanks

Answer 1

It is feasible, though I am unable to evaluate whether it is a good idea in your situation. Here's the practical answer however:

You could trigger and follow the pipeline run with a Azure CLI Task that runs in your Release stage. Azure CLI has Data Factory-specific commands which begin with az datafactory, so you can use them in both cases.

• starting the run with az datafactory pipeline-run
• waiting for its completion in a loop, running az datafactory pipeline-run show e.g. once a minute

Another solution could be using a REST API, such as in this example of monitoring the pipeline run

Answer 2

Is there any way to do this without any special trick in Az DevOps?

The direct answer is No cause the third-party task itself doesn't support this scenario by design.

According to comment from the Author liprec: At this moment the task only triggers a pipeline run and is not waiting for that run to complete. He has plans to add such a task to wait and poll the task run. So what you want could be possible in coming days, but for now it's not supported.

You have to use something like Powershell scripts to trigger ADF pipeline run via command-line like Mekki suggests above. Here's another similar PS example.