issue with CICD release pipeline in Azure Data Factory deployment

Question

I have created a release pipeline in Azure DevOps where I am moving the dev resources (pipeline, dataset, trigger, Integration runtime) to the Prod data factory.

Challenge is, In my dev ADF I am using an integration runtime say ir-myonpremdata that is self-hosted and shared, and the same ir is also present in prod data factory as self-hosted linked. Whenever I am deploying the pipelines I am updating the arm_templates for factory and JSON with the following

1.  ARMTemplateForFactory.JSON-code-

    {
        "name": "[concat(parameters('factoryName'), '/ir-myonpremdata')]",
        "type": "Microsoft.DataFactory/factories/integrationRuntimes",
        "apiVersion": "2018-06-01",
        "properties": {
            "type": "SelfHosted",
            "typeProperties": {
                "linkedInfo": {
                    "resourceId": "[parameters(**'ir-myonpremdata_properties_typeProperties_linkedInfo_resourceId'**)]",
                    "authorizationType": "Rbac"
                }
            }
        },
        "dependsOn": []
    }

2.  ARMTemplateParametersForFactory.json-

       
     "**ir-myonpremdata_properties_typeProperties_linkedInfo_resourceId**": {
            "type": "string",
            "defaultValue": "/subscriptions/d9368466-XXXX-4d83-XXXX-bb9f336fb6a7/resourcegroups/ModernDataPlatform/providers/Microsoft.DataFactory/factories/cpo-adf-dev/integrationruntimes/ir-myonpremdata"
        }

Then I am able to deploy the ADF otherwise it keeps throwing error messages. But it is very painful like if again someone will publish the master branch then in the adf_publish branch that extra piece of code will be removed automatically (as we added manually) from ARMTemplateForFactory.JSON and ARMTemplateParametersForFactory.json and the release pipeline will keep failing onwards.

I don't want to keep updating my JSON files whenever someone will publish from adf branch.

Are you using the steps in this document? Where did you update the arm_templates? — Jeff

Naga Pakalapati Naga Pakalapati · Accepted Answer · 2021-03-23T22:07:50

I have a similar issue... The best practice suggestion in this situation is to have a Shared IR on a separate ADF just to host IR and then use linked IR in all the dev/test/prod ADF's.

https://docs.microsoft.com/en-us/azure/data-factory/continuous-integration-deployment#best-practices-for-cicd

I am trying to do this ... will update whether it worked or not.

issue with CICD release pipeline in Azure Data Factory deployment

1 Answers