I'm trying to setup Istio 1.7 MultiCluster between Microk8s 1.18/Stable that is installed on Ubuntu 18.04 instance in Google Compute Engine and a GKE cluster. Everything is ok with GKE part. But I have a question regarding istio-ingressgateway on microk8s.
When I inspect services in the namespace "istio-system" of my Microk8s single-node cluster, I see, that "istio-ingressgateway" is stuck in "pending" state.
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/grafana ClusterIP 10.152.183.215 <none> 3000/TCP 10m
service/istio-egressgateway ClusterIP 10.152.183.180 <none> 80/TCP,443/TCP,15443/TCP 10m
service/istio-ingressgateway LoadBalancer 10.152.183.233 <pending> 15021:32648/TCP,80:30384/TCP,443:31362/TCP,15443:30810/TCP 10m
service/istiocoredns ClusterIP 10.152.183.70 <none> 53/UDP,53/TCP 10m
service/istiod ClusterIP 10.152.183.20 <none> 15010/TCP,15012/TCP,443/TCP,15014/TCP,853/TCP 10m
service/jaeger-agent ClusterIP None <none> 5775/UDP,6831/UDP,6832/UDP 10m
service/jaeger-collector ClusterIP 10.152.183.50 <none> 14267/TCP,14268/TCP,14250/TCP 10m
service/jaeger-collector-headless ClusterIP None <none> 14250/TCP 10m
service/jaeger-query ClusterIP 10.152.183.142 <none> 16686/TCP 10m
service/kiali ClusterIP 10.152.183.135 <none> 20001/TCP 10m
service/prometheus ClusterIP 10.152.183.23 <none> 9090/TCP 10m
service/tracing ClusterIP 10.152.183.73 <none> 80/TCP 10m
service/zipkin ClusterIP 10.152.183.163 <none> 9411/TCP 10m
Ok, I know that microk8s doesn't know that it is installed on the VM that is running inside GCP and thus can not create network loadbalancer in GCP like it can be easily done for service of type LoadBalancer in GKE. So I created LB manually (made it similar to the LB that GKE creates) and tried to attach it to the existing "istio-ingressgateway" service.
I ran:
kubectl edit svc -n istio-system istio-ingressgateway
And tried to put the IP of this LB in the same way and syntax as is see for istio-ingressgateway in GKE:
...
selector:
app: istio-ingressgateway
istio: ingressgateway
release: istio
sessionAffinity: None
type: LoadBalancer
status:
loadBalancer:
ingress:
- ip: 11.22.33.44
It doesn't work:
selector:
app: istio-ingressgateway
istio: ingressgateway
sessionAffinity: None
type: LoadBalancer
status:
loadBalancer: {}
So, my questions are:
- Is there a possibility to make Microk8s know that it is running on VM that is located in GCP and give it ability to create TCP LBs in "Network Services > LoadBalancing"? Maybe some annotation that can be added to the yaml of the service of type LoadBalancer?
- I found some info that if cloud infra doesn't support automated LB creation, then we can use host IP and NodePort of the istio-ingressgateway.
If the EXTERNAL-IP value is set, your environment has an external load balancer that you can use for the ingress gateway. If the EXTERNAL-IP value is (or perpetually ), your environment does not provide an external load balancer for the ingress gateway. In this case, you can access the gateway using the service’s node port.
But this was written not for MultiCluster setup. And for MultiCluster they suggest lusing of L4 LBs:
The IP address of the istio-ingressgateway service in each cluster must be accessible from every other cluster, ideally using L4 network load balancers (NLB). Not all cloud providers support NLBs and some require special annotations to use them, so please consult your cloud provider’s documentation for enabling NLBs for service object type load balancers. When deploying on platforms without NLB support, it may be necessary to modify the health checks for the load balancer to register the ingress gateway
is there a way to use NodePort for Istio MultiCluster setup between Microk8s (VM in GCE) and a GKE cluster?
Thanks a lot!
Pavel