azure data factory dependencies

Question

I have two activities in my azure data factory.

Activity A1 = a stored proc on a sql db. Input=none, output = DB (output1). The stored proc targets the output dataset.

Activity A2 = an azure copy activity ("type": "Copy") which copies from blob to same sql db. Input = blob, Output = DB (output2)

i need to run activity A1 before A2 and i cant for the world figure out what dependencies to put between them.

i tried to mark A2 as having two inputs - the blob + the DB (output1). if i do this, the copy activity doesn't throw error, but it does NOT copy the blob to db (i think it silently uses the DB as the source of copy, instead of blob as source of copy and somehow does nothing).

if i remove the DB input (output1) on A2 it can successfully copy blob to DB but i no longer have the dependency chain that A1 needs to run before A2

thanks!

mike mike · Accepted Answer · 2018-03-27T01:11:52

I figured this out - I was able to keep two dependencies on A2, but just needed to make sure of the ordering of the 2 inputs. Weird. Looks like the Copy activity just acts on the FIRST input - so when i moved the blob as the first input it worked! :) (earlier i had the DB output1 as first input and it silently did nothing)

 "activities": [
        {
            "type": "Copy",
            "typeProperties": {
                "source": {
                    "type": "BlobSource"
                },
                "sink": {
                    "type": "SqlSink",
                    "writeBatchSize": 0,
                    "writeBatchTimeout": "00:00:00"
                }
            },
            "inputs": [
                {
                    "name": "MyBlobInput"
                },
                {
                    "name": "MyDBOutput1"
                }
            ],
            "outputs": [
                {
                    "name": "MyDBOutput2"
                }
            ],
            "policy": {
                "timeout": "01:00:00",
                "concurrency": 3,
                "retry": 3
            },
            "scheduler": {
                "frequency": "Day",
                "interval": 1
            },
            "name": "AzureBlobtoSQL",
            "description": "Copy Activity"
        }
    ],

azure data factory dependencies

1 Answers