Azure Load Balancer with VM Scale Set: signal to remove VM from rotation

Question

I have a Virtual Machine Scale Set and a Load Balancer (regular LB, not Application Gateway). A health probe check an HTTP endpoint on the VM, which seems to work just fine: the endpoint returns a non-200 response if it knows it's not ready to process requests, which is a controlled and frequent state.

Problem: The lowest possible setting for probes are 5 seconds, and 2 consecutive failures, so at least 10 seconds will pass before a faulty VM is pulled from rotation. This is too long to wait since many requests could arrive and be rejected in that time period, even though the VM is aware of this state, and other VMs in the scale set are ready to process requests.

Question: Is there any way for the VM to instantly notify the load balancer that it wants to be pulled from rotation? The VM should then stay out of rotation until it starts returning 200 OK from the HTTP health probe endpoint.

Sidenote: az network nic ip-config address-pool remove (link) only seems to work for standalone VMs, not scale set VMs.

I have a question. You specify that VM Scale Set is used with a Load Balancer and that HTTP endpoint is used. Are you using regular Load Balancer or Application Gateway? The reason I'm asking is because HTTP endpoint probe is NOT the default mechanism on standard Load Balancer but it is the default on Application Gateway. Since you don't mention custom probe implementation I wanted to clarify this with you before I give an answer. Thank you. — Alex S
Hi @AlexS, thanks for asking- it's the regular load balancer, not app gateway. Would app gateway make a difference here? As far as I can tell, it still uses the regular load balancer under the covers. — bernhof

kim kim · Accepted Answer · 2018-03-13T21:20:15

Using PowerShell you can add a nic to a load balancer this way:

$lb= get-azurermloadbalancer -name NRP-LB -resourcegroupname NRP-RG
$backend=Get-AzureRmLoadBalancerBackendAddressPoolConfig -name LB-backend -LoadBalancer $lb
$nic =get-azurermnetworkinterface -name lb-nic1-be -resourcegroupname NRP-RG
$nic.IpConfigurations[0].LoadBalancerBackendAddressPools=$backend
Set-AzureRmNetworkInterface -NetworkInterface $nic

and in a similar way remove a nic from a LB

$nic = Get-AzureRmNetworkInterface -ResourceGroupName NRP-RG -Name lb-nic1-be
$nic.IpConfigurations[0].LoadBalancerBackendAddressPools = $null
Set-AzureRmNetworkInterface -NetworkInterface $nic

This way you can pull out your machine from the load balancer and add it back when its working again.

See e.g. this link for more details.

Azure Load Balancer with VM Scale Set: signal to remove VM from rotation

1 Answers