I'm trying to deploy an Azure AKS instance via ARM template.
I have a requirement to integrate the AKS instance into an existing Vnet.
I have a dedicated subnet for AKS service.
However, deployment has failed with the following error:
{"code":"DeploymentFailed","message":"At least one resource deployment operation failed.
Please list deployment operations for details. Please see
https://aka.ms/arm-debug for usage details.","details":
[{"code":"BadRequest","message":"{\r\n \"code\": \"InsufficientSubnetSize\",\r\n
\"message\": \"Pre-allocated IPs 93 exceeds IPs available in Subnet 11\",\r\n
\"target\": \"agentPoolProfile.count\"\r\n}"}]}
I'm using the following address space for Vnet: XX.XX.XX.0/24 (XX.XX.XX.0 - XX.XX.XX.255 which has 256 addresses.
I have a set of dedicated subnets within this Vnet, each of /28 mask (11+5 addresses depth):
XX.XX.XX.0/28
XX.XX.XX.16/28
XX.XX.XX.64/28
XX.XX.XX.128/28
XX.XX.XX.144/28
XX.XX.XX.160/28
XX.XX.XX.176/28
The subnet XX.XX.XX.144/28 is planned to be used in AKS.
The current AKS instance ARM template is as follows:
"resources": [
{
"type": "Microsoft.ContainerService/managedClusters",
"apiVersion": "2019-04-01",
"name": "[parameters('resourceName')]",
"location": "[parameters('location')]",
"dependsOn": [],
"tags": {},
"properties": {
"kubernetesVersion": "[parameters('kubernetesVersion')]",
"enableRBAC": "[parameters('enableRBAC')]",
"dnsPrefix": "[parameters('dnsPrefix')]",
"agentPoolProfiles": [
{
"name": "agentpool",
"osDiskSizeGB": "[parameters('osDiskSizeGB')]",
"count": "3",
"vmSize": "[parameters('agentVMSize')]",
"osType": "[parameters('osType')]",
"storageProfile": "ManagedDisks",
"maxPods": "30",
"vnetSubnetID": "/subscriptions/XXXXX/resourceGroups/XXXX/providers/Microsoft.Network/virtualNetworks/VNET_NAME/subnets/akssubnet"
}
],
"servicePrincipalProfile": {
"ClientId": "[parameters('servicePrincipalClientId')]",
"Secret": "[parameters('servicePrincipalClientSecret')]"
},
"networkProfile": {
"networkPlugin": "azure",
"serviceCidr": "10.0.0.0/16",
"dnsServiceIP": "10.0.0.10",
"dockerBridgeCidr": "172.17.0.1/16"
},
"addonProfiles": {
"httpApplicationRouting": {
"enabled": "[parameters('enableHttpApplicationRouting')]"
},
"omsagent": {
"enabled": "[parameters('enableOmsAgent')]",
"config": {
"logAnalyticsWorkspaceResourceID": "[parameters('omsWorkspaceId')]"
}
}
}
}
},
"subscriptionId": "[split(parameters('omsWorkspaceId'),'/')[2]]",
"resourceGroup": "[split(parameters('omsWorkspaceId'),'/')[4]]"
}
]
Network profile parameters were set according to the following article: Microsoft.ContainerService managedClusters template reference
A CIDR of 10.0.0.0/16 is of a private range and isn't interfering with my existing Vnet range.
I need advice on how to deal with this deployment error.
Upd:
I've tried the deployment with the values of my Vnet/subnets but stil it's failing:
Upd2:
Per MS documentation "Minimum number of pods on the initial cluster creation using Azure CNI type is 30" which leads to the following number of subnet range in my case according to the formula: (number of nodes + 1) + ((number of nodes + 1) * maximum pods per node that you configure) = (3+1) + ((3+1)*30) = 124
So the multiplier of 30 will be always present even if the number of pods is set to 1 in ARM template for example.
Upd3:
However, as I was unable to extend the existing subnet range I've managed to deploy the AKS instance using the following configuration:
"parameters": {
"SvcCidr": {
"type": "string",
"defaultValue": "10.0.0.0/16",
"metadata": {
"description": "Maximum number of pods that can run on a node."
}
},
"PodCidr": {
"type": "string",
"defaultValue": "10.244.0.0/16",
"metadata": {
"description": "Maximum number of pods that can run on a node."
}
},
"DnsSvcIP": {
"type": "string",
"defaultValue": "10.0.0.10",
"metadata": {
"description": "Maximum number of pods that can run on a node."
}
},
"DockerCidr": {
"type": "string",
"defaultValue": "",
"variables": {
"vnetSubnetId": "[resourceId(resourceGroup().name, 'Microsoft.Network/virtualNetworks/subnets', parameters('vnetName'), parameters('vnetSubnetName'))]",
"resources": [
{
"type": "Microsoft.ContainerService/managedClusters",
"agentPoolProfiles": [
{
"vnetSubnetID": "[variables('vnetSubnetId')]",
"networkProfile": {
"networkPlugin": "[parameters('NetPlugin')]",
"serviceCidr": "[parameters('SvcCidr')]",
"podCidr": "[parameters('PodCidr')]",
"DNSServiceIP": "[parameters('DnsSvcIP')]",
"dockerBridgeCidr": "[parameters('DockerCidr')]"
Which leads to the provision of my subnet range IP addresses only to cluster nodes while the pods will use the private IP addresses range.
