I am trying to setup an Azure Data Factory in a CI/CD setup. I followed Microsoft's best practices (https://docs.microsoft.com/en-us/azure/data-factory/continuous-integration-deployment).
I have 4 environments (dev, test, UAT, PRD)
In short what I have done:
Create an Azure data factory and link it to my Azure DevOps repo on DEV environment
Create an Azure data factory on the other environments (test, UAT and PRD), but do NOT link it to DevOps. Instead, pipelines are released on these data factories using ARM templates and release pipelines in Azure DevOps.
I have parameterized all necessary parts to be able to overwrite the settings in each of my environments.
At this moment, I am able to succesfully deploy to my other environments, however, the linkedservice to my database on azure is not working. I have parameterized everything, like Microsoft suggests, but when I export my linkedservice to an ARM template, it uses a connection string instead of my parameterized settings.
Below is a picture of my settings in the Azure Data Factory portal:
When I try to export this to ARM templates, I get the following:
{
"name": "[concat(parameters('factoryName'), '/database')]",
"type": "Microsoft.DataFactory/factories/linkedServices",
"apiVersion": "2018-06-01",
"properties": {
"parameters": {
"sqlServerUrl": {
"type": "string"
},
"databaseName": {
"type": "string"
},
"sqlPwd": {
"type": "string"
},
"sqlAdminUsername": {
"type": "string"
}
},
"annotations": [],
"type": "AzureSqlDatabase",
"typeProperties": {
"connectionString": {
"type": "SecureString",
"value": "[parameters('database_connectionString')]"
}
}
},
"dependsOn": []
},
The problem with this ARM templates is that it does not use the parameters to create the connection string, but it uses the connection string parameter database_connectionString (the connection string is by default always parameterized by Azure, so I cannot remove this parameter).
When the release pipeline uses this template, the connectionstring is not filled in (only the parameters are filled in) and hence, the connection to the database fails. How should you setup the connection so that you can automatically (without human interaction) deploy to all environments by only changing the parameters?
I do not want to change the ARM templates coming from Azure Data Factory, because this requires human interaction