Capture output in Azure Data Factory WebActivity

0
votes

We have a Pipeline that invokes an API endpoint using WebActivity to start a long-running job. As per specification, the API starts the job in the background and returns HTTP Status Code 202 along with a json object like this:

{
    "TransactionID": "534b1867-a5ac-41b3-9a57-xxx",
    "TransactionStatus": "Processing"
}

Then the Pipeline waits until the status of that TransactionID changed to Completed using a combo of another WebActivity and WaitActivity in a Loop activity.

This pipeline was working correctly until today (June, 29th 2020) but now the json output of the WebActivity is not captured anymore by ADF. Before anyone asks: we have not changed anything in the Pipeline or in the API itself, it just stopped working.

My question is: why Azure Data Factory is not capturing anymore the output of web activities when the status code is not 200? Were we doing somethign wrong and now Azure fixed it or has Azure changed the behaviour of WebActivity without notice?

1
azure-data-factory

1 Answers

0
votes

Not surprisingly, Azure Data Factory changed behavior today (June 30) and all pipelines started working again.

If you came here to see a solution, there isn't any. It solved by itself.

I will keep this post in case it happens again so we can discuss and update accordingly.