I remember DNS records being cached locally on various linux distros in the past, but this appears to have changed over the years (DNS caching in linux).
Within our environment (non-K8S) we found a noticeable delay (1-2ms) for each request due to DNS lookups because of this.
I also noticed there is no local DNS cache within K8S by default (https://github.com/kubernetes/kubernetes/issues/45363) and the DNS cache within CoreOS is also disabled by default (https://coreos.com/os/docs/latest/configuring-dns.html).
Given we're considering migrating towards K8S I was wondering; why is this not enabled for Kubernetes in particular?
My only theory is within kube-dns records are updated pre-emptively to ensure high-availability; but I'm not sure if K8S actually does that?
As a workaround, if I were to run dnsmasq on every node, would I break things? I noticed there have been attempts to make that setup the default within K8S, but those attempts/PR's appear to have gone stale and I'm not sure why.