How to delete an Azure Table after it is copied to another storage using ADF

Question

I have 100 tables which i want to copy to another storage account frequently. After the copy is over, i want to delete the source tables. I am able to copy entities inside tables to another storage account using ADF Copy Activity. But couldn't figure out a way to delete the source tables after successful copy.

I am using DataFactory .NET API to create pipelines, datasets etc. I thought of Custom Activity as the solution but not sure how to plug this actvity into pipeline through API ?

Any code samples or suggestions are highly appreciated.

Tom Sun - MSFT Tom Sun - MSFT · Accepted Answer · 2017-06-08T03:20:49

As metioned that we could do that with Custom Activity.

but not sure how to plug this actvity into pipeline through API?

We could use the create or update Pipeline API to create or update the pipeline API.

We could get more info about how to use custom activities in an Azure Data from this tutorials.

The following is the snippet from the tutorials.

1.Create a custom activity .NET Class Library project implements that IDotNetActivity interface

2.Launch Windows Explorer, and navigate to bin\debug or bin\release folder

3.Zip the all of the file under bin\release folder and upload to the azure storage customactivitycontainer

4.Create Azure Storage linked service

5.Create Azure Batch linked service

We could use the create or update Pipeline API to create a pipeline that uses the custom activity

{
  "name": "ADFTutorialPipelineCustom",
  "properties": {
    "description": "Use custom activity",
    "activities": [
      {
        "Name": "MyDotNetActivity",
        "Type": "DotNetActivity",
        "Inputs": [
          {
            "Name": "InputDataset"
          }
        ],
        "Outputs": [
          {
            "Name": "OutputDataset"
          }
        ],
        "LinkedServiceName": "AzureBatchLinkedService",
        "typeProperties": {
          "AssemblyName": "MyDotNetActivity.dll",
          "EntryPoint": "MyDotNetActivityNS.MyDotNetActivity",
          "PackageLinkedService": "AzureStorageLinkedService",
          "PackageFile": "customactivitycontainer/MyDotNetActivity.zip",
          "extendedProperties": {
            "SliceStart": "$$Text.Format('{0:yyyyMMddHH-mm}', Time.AddMinutes(SliceStart, 0))"
          }
        },
        "Policy": {
          "Concurrency": 2,
          "ExecutionPriorityOrder": "OldestFirst",
          "Retry": 3,
          "Timeout": "00:30:00",
          "Delay": "00:00:00"
        }
      }
    ],
    "start": "2016-11-16T00:00:00Z",
    "end": "2016-11-16T05:00:00Z",
    "isPaused": false
  }
}

About how to operate azure storage table please refer to document.

How to delete an Azure Table after it is copied to another storage using ADF

1 Answers