Azure Data Factory WebHook Fails after 1 minute

3
votes

We have a few Azure Functions that calls an API endpoint that takes >230 seconds (the maximum runtime for Azure Function call from ADF). The work around we found was to use the Webhook activity and using the callBackUri. But for whatever reason, the webhook always fails at 00:01:01 with a BadRequest Error:

BadRequestError:
BadRequestError

If the function completes within that minute, the callback is working correctly and runs fine.

The WebHook's Timeout is set to 10 minutes (00:10:00), but after 1 minute it will raise a BadRequest error. The Azure function continues to run in the background and will successfully complete it's task, but my pipeline is now broken and not continue to the next step.

I cannot use Durable Azure Functions, as that is not yet supported in Python Azure Functions.

2
azure-data-factory
The default timeout for Web activity or Azure functions activity appears to be of 7 days so not sure if webhook workaround was required. Was there any specific error you faced related to timeout when you tried inovkoking your azure function with web activity or with azure functions activity in the ADF with default timeout setting configured? – Bhushan
The docs states "Azure Functions times out after 230 seconds regardless of the functionTimeout setting" hence the need for the work around and recomendation for Durable Functions link – Andrew Buttigieg

2 Answers

2
votes

After some further invetigation, the 1 min timeout error is expected. Reading the docs, for long running calls, the activity expects a 202 (Accepted) responce within the minute.

https://docs.microsoft.com/en-us/azure/data-factory/control-flow-webhook-activity#additional-notes

Details on the Asynchronous Request-Reply pattern (and sample c# code) is avalable here: https://docs.microsoft.com/en-us/azure/architecture/patterns/async-request-reply

1
votes

I managed to use a work around in the Azure Function itself by using -Timeout param. Details here: Using Azure Data Factory's Web Hook Activity to invoke/poll Azure Powershell Function App times out after one minute

Sample running code below:

$Body = @{
            callbackUri = $Request.Body.callBackUri;
        } | ConvertTo-Json
try{
    # This API will be responsible for issuing the call back after it has finished the long running process
    $output = Invoke-RestMethod -Method Post -Body $Body -Uri $funcapp2Url -ContentType 'application/json' -TimeoutSec 1
}
catch {
    Write-Output $_
}

# Return HTTP 202 immediately
Push-OutputBinding -Name Response -Value ([HttpResponseContext]@{
    StatusCode = [HttpStatusCode]::Accepted
    Body = "Wait for callback"
}) -Clobber