Azure Data Factory V2 Copy activity Mapping deployment issue

Question

Consider following test Mapping for Data Factory Copy activity:

"translator": {
  "columnMappings": "@json('{\"from\":\"to\"}')",
  "type": "TabularTranslator"
}

After deploying pipeline with the help of Set-AzureRmDataFactoryV2Pipeline PowerShell cmdlet we get normally deployed pipeline with the exact columnMappings value as specified in source code. But if you try to be more dynamic:

"translator": {
  "columnMappings": "@json(pipeline().parameters.Mapping)",
  "type": "TabularTranslator"
}

then after deployment you'll find that translator element is completely missing in pipeline. A workaround - set translator in Azure Portal Data Factory pipeline editing UI (either in Designer or JSON modes - both options work). But if after these manipulations you save pipeline JSON to the file and attempt to deploy it via Set-AzureRmDataFactoryV2Pipeline PowerShell cmdlet - bang, translator becomes missing. Expected result - deployment shall preserve translator element, because Portal JSON Editor preserves it.

We are doing automated deployment of pipelines (as you already figured out - with the help of Set-AzureRmDataFactoryV2Pipeline) and this bug breaks our automated deployment because it requires manual postdeployment pipeline editing on Azure Portal UI.

What may be the reason of such a buggy behavior? Can you suggest an idea how to work around this bug in automated way, or how to fix the code so it can be properly deployed with Set-AzureRmDataFactoryV2Pipeline?

For now I've resolved an issue by deploying this particular pipeline that has translator property with the help of ARM templite and New-AzureRmResourceGroupDeployment PowerShell cmdlet. Moving deployment of all Data Factory objects from PowerShell to ARM is also an option, but not an easy. — Roman

Fang Liu Fang Liu · Accepted Answer · 2018-11-17T10:51:12

You could try whether "Update-Module -Name AzureRm.DataFactoryV2" helps. It might be caused by that your powershell module is out of date.

Azure Data Factory V2 Copy activity Mapping deployment issue

1 Answers