azure kubernetes-internal load balancer access by dns name instead of ip address

Question

problem statement

we are planning to use azure api management service as a reverse proxy for our AKS . I took reference of following URL for configuring azure api manager with AKS. Although it gives information about node port but same can be applied through internal load balancer IP address.

https://fizzylogic.nl/2017/06/16/how-to-connect-azure-api-management-to-your-kubernetes-cluster/

we are currently having multiple environments such as dev1,dev2, dev3, dev, uat,stage, prod. we are trying to automate this configuration step and dont need to bind to specific IP but need to point to dns name associated with internal load balancer fro k8s.

Answer 1

If you use annotation on the service to use an internal loadbalancer you will get an IP Address on the vNet for your Service rather than an external IP.

annotations: service.beta.kubernetes.io/azure-load-balancer-internal: "true"

You can then use the external-dns service (https://github.com/kubernetes-incubator/external-dns) to automatically create DNS entries for your services inside Azure DNS zones. You should then be able to resolve to the service DNS name.

Although not explicitly supported, it does work with Private DNS zones as well.

Answer 2

Part of the problem is answered by @Ben. I would caution on using external-dns open source as you may not like to create dependency on this very important function. It requires you to grant additional permission!

You will need a virtual private ip and it's achieved by internal load balancer annotation and it works. I recently documented end of end tls/ssl with internal load balancer and can find it at https://blogs.aspnet4you.com/2019/01/06/end-to-end-tlsssl-offloading-with-application-gateway-and-kubernetes-ingress/.

Keep in mind, my solution worked great until I removed http application routing add-on. Why? The add-on came with Azure Dns (public) and public load balancer. Both of them are removed for good when I removed the add-on but the removal broke the dns entry associated with vip of internal load balancer. I didn't intend to remove dns zone. My attempt to create new DNS Zone and add A record with private IP didn't work. Kubernetes can't resolve the fqdn. Tried with Azure Private DNS but it's not able to resolve either! My attempt to use configmap with kube-dns didn't work and it breaks dns resolution of other things if I included upstream! So, investigation continues!

I would love to hear how you solved the fqdn problem.

On the optimistic note, I think VM based custom dns server can be good option and you would likely have one for hybrid solution.

Answer 3

It should be possible to configure Azure VNET to rely on your own DNS server: https://docs.microsoft.com/en-us/azure/virtual-network/virtual-networks-name-resolution-for-vms-and-role-instances#name-resolution-that-uses-your-own-dns-server. ONce setup APIM will use it to resolve addresses for all outgoing requests. Just make sure that you would support resolving public addresses as well, at there are few dependencies to APIM without which it will not work.