Using Istio, is there a way to route ingress to a k8s cluster service that is not running an envoy sidecar?

1
votes

I'm running a Azure AKS cluster with both Windows and Linux VMs.

I can curl the cluster service by name from a pod in the Istio namespace, so I know TCP to the pod works. I believe I need to inform my Virtual Service in some way to not route through the envoy proxy, but just forward requests directly to the k8s service endpoint - similar to as if it were a VM external to the mesh. I do have TLS terminating at the gateway - the k8s service itself is just exposed inside the cluster on port 80.

Currently, there is no envoy sidecar for Windows containers, but from k8s perspective, this is just another service in the same cluster Istio is running in.

http-gateway.yaml 

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  annotations:
  name: http-gateway
  namespace: istio-system
spec:
  selector:    
    istio: ingressgateway
  servers:
  - hosts:
    - "*.myapp.com"
    port:
      number: 80
      name: http-80-gateway
      protocol: HTTP
    tls:
      httpsRedirect: true # sends 301 redirect for http requests
  - hosts:
    - "*.myapp.com"
    port:
      number: 443
      name: https-443-gateway
      protocol: HTTPS
    tls:
      credentialName: cert-azure-dns
      privateKey: sds
      serverCertificate: sds
      mode: SIMPLE

virtual-service.yaml

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: myapp-vsvc
  namespace: myapp-ns
spec:
  hosts:
  - foo #external DNS name is foo.myapp.com; gateway host for HTTPS is '*.myapp.com'
  gateways:
  - istio-system/http-gateway
  http:
  - route:
    - destination:
        host: myapp-svc.myapp-ns.svc.cluster.local
        port:
          number: 80

Attempting an Envoy Passthrough I've added a ServiceEntry like the following:

apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: myapp-se
  namespace: myapp-ns
spec:
  hosts:
  - myapp-svc.myapp-ns.svc.cluster.local
  ports:
  - number: 80
    name: http
    protocol: HTTP
  resolution: DNS
  location: MESH_EXTERNAL

The server response is a 404 with a "server" header value of "istio-envoy".

DNS is resolving to the gateway correctly and the acme cert is valid - so this error usually indicates I've made it to the Virtual Service, but haven't been routed to a cluster service. In Kiali, there are no Istio validation errors on any of my yaml definitions: virtual service, service entry or gateway.

My global.outboundTrafficPolicy.mode is set to "ALLOW_ANY".

I wonder if declaring "EXTERNAL_MESH" for a cluster service is a problem? Istio knows the k8s service exists, so is it trying to give priority to routing to the envoy sidecar and ignoring my service entry registration?

There is an option to bypass envoy altogether for specific IP ranges, which would be an option if I could somehow set a static IP on this particular cluster service. I want to bypass envoy for ingress to this one cluster service.

2
azurekubernetesistioazure-aks

2 Answers

1
votes

you would use the serviceentry to do that:

apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: httpbin-ext
spec:
  hosts:
  - httpbin.org
  ports:
  - number: 80
    name: http
    protocol: HTTP
  resolution: DNS
  location: MESH_EXTERNAL

https://istio.io/docs/tasks/traffic-management/egress/egress-control/

1
votes

I could have sworn I tried this before, but apparently all I needed to provide was a simple Virtual Service without any Destination Rule or Service Entry. 

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: myapp-vsvc
  namespace: myapp-ns
spec:
  hosts:
  - foo.myapp.com 
  gateways:
  - istio-system/http-gateway
  http:
  - route:
    - destination:
        host: myapp-svc.myapp-ns.svc.cluster.local
        port:
          number: 80