Exposing virtual service with istio and mTLS globally enabled

Question

I've this configuration on my service mesh:

mTLS globally enabled and meshpolicy default
simple-web deployment exposed as clusterip on port 8080
http gateway for port 80 and virtualservice routing on my service

Here the gw and vs yaml

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: http-gateway
spec:
  selector:
    istio: ingressgateway # Specify the ingressgateway created for us
  servers:
  - port:
      number: 80 # Service port to watch
      name: http-gateway
      protocol: HTTP
    hosts:
    - "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: simple-web
spec:
  gateways:
  - http-gateway
  hosts:
  - '*'
  http:
  - match:
    - uri:
        prefix: /simple-web
    rewrite:
      uri: /
    route:
    - destination:
        host: simple-web
        port:
          number: 8080

Both vs and gw are in the same namespace. The deployment was created and exposed with these commands:

k create deployment --image=yeasy/simple-web:latest simple-web
k expose deployment simple-web --port=8080 --target-port=80 --name=simple-web

and with k get pods I receive this:

pod/simple-web-9ffc59b4b-n9f85   2/2     Running

What happens is that from outside, pointing to ingress-gateway load balancer I receive 503 HTTP error. If I try to curl from ingressgateway pod I can reach the simple-web service. Why I can't reach the website with mTLS enabled? What's the correct configuration?

I will but I think it is a common problem. Here there is something similar stackoverflow.com/questions/54160215/… — Manuel Castro
well, I have deployed bunch of istio apps with mutual tls, and it almost always works for me. — suren
all internal routing seems to work fine, the problem is only with reachability from ingress gateway — Manuel Castro
Hi, what enviroment and versions of kubernetes, istio and network pluggin do you have? — Piotr Malec

Unknown Unknown · Accepted Answer · 2019-10-15T16:50:39

As @suren mentioned in his answer this issue is not present in istio version 1.3.2 . So one of solutions is to use newer version.

If you chose to upgrade istio to newer version please review documentation 1.3 Upgrade Notice and Upgrade Steps as Istio is still in development and changes drastically with each version.

Also as mentioned in comments by @Manuel Castro this is most likely issue addressed in Avoid 503 errors while reconfiguring service routes and newer version simply handles them better.

Creating both the VirtualServices and DestinationRules that define the corresponding subsets using a single kubectl call (e.g., kubectl apply -f myVirtualServiceAndDestinationRule.yaml is not sufficient because the resources propagate (from the configuration server, i.e., Kubernetes API server) to the Pilot instances in an eventually consistent manner. If the VirtualService using the subsets arrives before the DestinationRule where the subsets are defined, the Envoy configuration generated by Pilot would refer to non-existent upstream pools. This results in HTTP 503 errors until all configuration objects are available to Pilot.

It should be possible to avoid this issue by temporarily disabling mTLS or by using permissive mode during the deployment.

Exposing virtual service with istio and mTLS globally enabled

2 Answers