Calling a singular Logic app from multiple steps in a pipeline

Question

I have created an ADF pipeline which has several components to in it; execute a stored procedure, perform a copy data task. There are 14 in total (7 pair) and I want to trigger a failure that will send out an email with the error message.

I've gone and created a logic app to send an email as described in this link http://microsoft-bitools.blogspot.com/2018/03/add-email-notification-in-azure-data.html

In the 'Web Activity' component-> Settings-> Body, I have the following:

    "DataFactoryName":
        "@{pipeline().DataFactory}",
    "PipelineName":
        "@{pipeline().Pipeline}",
    "ErrorMessage":
        "@{activity('Execute Package').error.message}",
    "EmailTo":
        "@{pipeline().parameters.EmailTo}"
}

The 'Execute Package' is the name of the Stored Procedure or Copy Data activity.

The problem is that it only works for the named activity 'Execute Package'. I haven't been able to find anywhere that it can dynamically get where it is coming from. Is there any way to just have a singular web activity? I don't want to create 14 more things in my pipeline each to handle a different possible failure. The call to SendCompletionEmail works fine with the logic app since only one thing is calling it.

MartinJaffer-MSFT MartinJaffer-MSFT · Accepted Answer · 2020-01-15T01:48:39

I'd like to take a moment to point out that after solving the immediate issue (how to get the error out of multiple sources), there is another issue you will face: The FailureSendEmail activity will not run when you expect.

First issue (how to get error of whichever is failing): Assuming that your pipeline is linear, and you only expect one activity to fail, and all subsequent to not run, I recommend you use the coalesce function. Coalesce grabs the first non-null argument. Here is an example I worked out using 2 stored procs: @string(coalesce(activity('Stored Procedure1').Error,activity('Stored Procedure2').Error)) Coalesce takes an arbitrary number of arguments, so you can expand this as you like.

The other issue is all the 'on failure' connections. While what you built makes sense, please let me explain. When an activity has multiple dependency connections coming into it, it will not execute unless all the dependencies report in. These are AND'ed, not OR'ed.

Fortunately, you do not need a direct connection in order to reference the activity outputs/errors. An indirect connection is all you need. If your pipeline logic is linear, then you already have this. Remove all the lines going into FailureSendEmail. Then add an 'on failure' and a 'skipped' dependency connection from your last 'Copy Data' activity. The logic goes like this:

Assuming all the activities in your 'happy path' are connected by success dependencies, If one activity fails early in the pipeline, then the subsequent activities are skipped. This fulfills the skipped dependency. If the last copy activity fails, this fulfills the failure dependency.

Calling a singular Logic app from multiple steps in a pipeline

1 Answers