Istio: failed calling admission webhook Address is not allowed

1
votes

I am getting the following error while creating a gateway for the sample bookinfo application

Internal error occurred: failed calling admission webhook "pilot.validation.istio.io": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: Address is not allowed

I have created a EKS poc cluster using two node-groups (each with two instances), one with t2.medium and another one is with t2.large type of instances in my dev AWS account using two subnets with /26 subnet with default VPC-CNI provided by EKS

But as the cluster is growing with multiple services running, I started facing issues of IPs not available (as per docs default vpc-cni driver treat pods as an EC2 instance)

to avoid same I followed following post to change networking from default to weave

https://medium.com/codeops/installing-weave-cni-on-aws-eks-51c2e6b7abc8

because of same I have resolved IPs unavailability issue,

Now after network reconfiguration from vpc-cni to weave

I am started getting above issue as per subject line for my service mesh configured using Istio

There are a couple of services running inside the mesh and also integrated kiali, prometheus, jaeger with the same.

I tried to have a look at Github (https://github.com/istio/istio/issues/9998) and docs (https://istio.io/docs/ops/setup/validation/), but could not get a proper valid answer.

Let me if anyone face this issue and have partial/full solution on this.

1
amazon-web-serviceskubernetesistioamazon-eks

1 Answers

2
votes

This 'appears' to be related to the switch from AWS CNI to weave. CNI uses the IP range of your VPC while weave uses its own address range (for pods), so there may be remaining iptables rules from AWS CNI, for example.

Internal error occurred: failed calling admission webhook "pilot.validation.istio.io": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: Address is not allowed

The message above implies that whatever address istio-galley.istio-system.svc resolves to, internally in your K8s cluster, is not a valid IP address. So I would also try to see what that resolves to. (It may be related to coreDNS).

You can also try the following these steps;

Basically, (quoted)

  • kubectl delete ds aws-node -n kube-system
  • delete /etc/cni/net.d/10-aws.conflist on each of the node
  • edit instance security group to allow UDP, TCP on 6873, 6874 ports
  • flush iptables, nat, mangle, filter
  • restart kube-proxy pods
  • apply weave-net daemonset
  • delete existing pods so the get recreated in Weave pod CIDR's address-space.

Furthermore, you can try reinstalling everything from the beginning using weave.

Hope it helps!