How to stop scheduled azure datafactory job (slice) runs

2
votes

A while ago I created a simple job that copied data from blob storage to an azure data lake. I created a schedule for it to run every day. Since it was an experiment I want to stop the job from running every day. In order to do that I have tried the following

  • Pause the job
  • Delete the pipeline

But both did not prevent the job from running every day. I know that because I set an alert on the job activity, and I still receive a daily email.

In order to minimize the work I now set the source blob input to an invalid path so there is no actual transfer of data but I am still wondering what needs to be done to completely stop all activity.

EDIT: this is the alert I get daily:

RULE NAME: CL

RULE DESCRIPTION:

OPERATION NAME: RunFinished

EVENT DESCRIPTION: In data factory centralelogging, validation for table InputDataset-cuv, slice start time 2017-07-19T00:00:00.0000000Z, finished at 2017-07-20T00:00:26.1308306Z.

OPERATION ID:

RESOURCE URI: /subscriptions/ff346b9c-1111-44d4-8668-354e5063cf18/resourcegroups/dataanalysis/providers/Microsoft.DataFactory/datafactories/centralelogging

SUBSCRIPTION NAME: Visual Studio Enterprise with MSDN

PROPERTIES: TableName(s)=InputDataset-cuv, SliceStartTime=2017-07-19T00:00:00.0000000Z, SliceEndTime=2017-07-20T00:00:00.0000000Z, BatchTime=2017-07-20T00:00:00.0000000Z, RetryCount=0, RunStartTime=2017-07-20T00:00:03.2144867Z, DataFactoryName=centralelogging

You will be alerted whenever an event matches this filter: OperationName = 'RunFinished', ResourceUri = '/subscriptions/ff346b9c-1111-44d4-8668-354e5063cf18/resourcegroups/dataanalysis/providers/Microsoft.DataFactory/datafactories/centralelogging', Status = 'Failed'

As previously stated to stop execution I set the source blob input to an invalid path and when I click the email and zoom in to the details I can see that the job fails.

2
azureazure-data-factory
If you've deleted the pipeline then that Data Factory job no longer exists and would not be running. Did you create some Azure Automation also? Is it that part that still exists and is notifying you. Consider deleting your Azure Automation section relevant to this job. - wBob
You can set Paused:True in pipeline activity. - Venky
@Venky I did, but it did not help. In fact, I have deleted the whole pipeline but I still get mail alerts - Peter Bons
@wBob I did not do anything with Azure Automation - Peter Bons
Could this be happening from another Azure subscription? Sounds strange - consider opening a helpdesk request via the Azure portal and report back here for the benefit of future users. - wBob

2 Answers

0
votes

Apparently the schedule is not stored in the pipeline but in the source dataset. Deleting the pipeline is therefore not enough, I deleted the dataset and now the alerts are gone.

0
votes

Follow the below steps to pause/stop pipeline in Azure portal:

  1. Go to "Monitoring & Manage" section of Azure Data Factory and choose the pipeline int he left side.
  2. You can pause the pipeline. Currently running slices will still run to completion. New slices will not start.
  3. You can stop the pipeline. Currently running slices will also be stopped and any more slices will not start.