How do I raise an error from if activity in ADFV2?

Question

I have a child pipeline that receives instructions from a control file using a dataset. These instructions define what directory to copy files from.

First this child pipeline checks that a file exists via Get Metadata activity on the source folder. It then executes a child pipeline to process the data if one or more childitems are returned from GetMetaData.

In the control dataset, there is also a required Y/N field meaning I can ignore the error if the folder or file does not exist.

If the folder does not exist, the GetMEtadata will fail. If it exists but no files, I get 0 child items. So 2 different things happen for file or folder missing (an error or 0 items).

In either case, I need to route the output of GetMetaData to an IF that checks if the file was required. If not, consume the error and return. If it was required, raise an error.

I can't find a way to raise an error though. Just as important, is there an alternative approach that would work or fit better with ADF V2 design?

Many thanks, Mark.

Answer 1

I use a Set Variable Task to achieve this.

In the variable expression, I divide by zero. This throws an error which propigates up my pipeline(s) as needed. Most generic way I could think to do it.

"name": "Throw Error",
"description": "You can't throw an error in DF so I try to divide by zero. :)",
"type": "SetVariable",
"dependsOn": [],
"userProperties": [],
"typeProperties": {
   "variableName": "ThrowError",
   "value": {
      "value": "@div(1,0)",
      "type": "Expression"
   }
}

Answer 2

i have the similar scenario and i manage that with SqlStoreProcedure.

"storedProcedureName": "sp_executesql", "storedProcedureParameters": { "stmt": { "value": "Declare @err_msg NVARCHAR(150)SET @err_msg=N'Error occurred in this pipeline somehow somewhere something. Best regards. EXISTSCheers'RAISERROR(@err_msg,15,1)", "type": "string" } }

StoredProcedureName: sp_executesql

StoredProcedureParameter: stmt

value: Declare @err_msg NVARCHAR(150)SET @err_msg=N'Error occurred in this pipeline somehow somewhere something. Best regards. EXISTSCheers'RAISERROR(@err_msg,15,1) (of course you can change error text :)

type: string

Hope this helps

Answer 3

I have found the easiest way to throw an error is to do a SQL lookup on the following query:

THROW 99999, 'Error message thrown', 1

For us, we make extensive use of Azure Function Apps, so we created a special function called "ThrowError" that throws an error with a message provided as a parameter. That is the easiest only if you already use Azure Function Apps, not worth it to just create the one function.

Answer 4

The solution I am using is to create a mock API in API Management that returns 500 HTTP code together with an error message. More details on how to do this can be found here: https://docs.microsoft.com/en-us/azure/api-management/mock-api-responses.

From inside the ADF pipeline, I call this mock API using Web Activity passing in the error message.

Then I can see the failed pipeline with expected error.

Answer 5

I use a Web Activity and GET a URL that doesn't exist: www.notreal-nvlsuifhg9348h4932.com.

(I do like Chris Hume's Set 1/0 option though. Might steal that ;) )