Kubernetes internal nginx ingress controller with SSL termination & ssl-passthrough

Question

I am very new to using helm charts for deploying containers, and I have also never worked with nginx controllers or ingress controllers. However, I am being asked to look into improving our internal nginx ingress controllers to allow for SSL-passthrough.

Right now we have external (public facing) and internal controllers. Where the public ones allow SSL-passthrough, and the internal ones have SSL-termination. I have also been told that nginx is a reverse proxy, and that it works based on headers in the URL.

I am hoping someone can help me out on this helm chart that I have for the internal ingress controllers. Currently I am under the impression that having SSL termination as well as SSL-passthrough on the same ingress controllers would not be possible. Answered this one myself: https://serversforhackers.com/c/tcp-load-balancing-with-nginx-ssl-pass-thru

Our current (internal) ingress code:

---

rbac:
  create: true

controller:
  ingressClass: nginx-internal
  service:
    annotations:
      service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0
      service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:eu:110:certificate/62-b3
      service.beta.kubernetes.io/aws-load-balancer-ssl-ports: !!str 443
      service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
      service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: !!str 3600

    targetPorts:
      https: 80

  replicaCount: 3

defaultBackend:
  replicaCount: 3

Can I simply add the following? :

controller:
    extraArgs:
        enable-ssl-passthrough: ""

Note: The above piece of code is what we use on our external ingress controller.

additionally, I found this: Ingress and SSL Passthrough

Can I just go and mix the annotations? Or do annotations only care about the 'top domain level' where the annotation comes from? eg:

service.beta.kubernetes.io
nginx.ingress.kubernetes.io

Both come from the domain kubernetes.io, or does the sub-domain make a difference? I mean: https://github.com/kubernetes/ingress-nginx/blob/master/docs/user-guide/nginx-configuration/annotations.md That page doesn't show any of the service.beta annotations on it ..

What's the difference between the extraArg ssl-passthrough configuration and the ssl-passthrough configuration in the annotations?

I'm looking mostly for an answer on how to get the SSL-passthrough working without breaking the SSL-termination on the internal ingress controllers. However, any extra information to gain more insight and knowledge as far as my other questions go would also be very appreciated :)

Marco Marco · Accepted Answer · 2019-11-15T06:51:03

So I found the answer to my own question(s): The annotations appear to be 'configuration items'. I'm using quotes because i can't find a better term. The extraArgs parameter is where you can pass any parameter to the controller as if it were a commandline parameter. And I think it is also safe to say that the annotations can be either any of the same top-level domain. I have not found any that weren't from another domain then kubernetes.io

To get my ingress controller to work side-by-side with the SSL-termination controller the helm chart looks as following:

---
rbac:
  create: true

controller:
  ingressClass: nginx-internal-ssl-passthrough
  service:
    annotations:
      nginx.ingress.kubernetes.io/ssl-passthrough: "true"
      service.beta.kubernetes.io/aws-load-balancer-type: nlb
      service.beta.kubernetes.io/aws-load-balancer-internal: "true"
      service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true"
      service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags: "tag3=value3, tag3=value3, tag3=value3, tag3=value3"
    targetPorts:
      https: 443
  replicaCount: 2
  extraArgs:
    enable-ssl-passthrough: ""

defaultBackend:
  replicaCount: 2

Toke me about 2 days of researching/searching the web & 6 deployments to get the whole setup working with AWS nlb, ssl-passthrough enabled, cross-zone loadbalancing, etc. But after having found the following pages it went pretty fast: https://kubernetes.github.io/ingress-nginx/deploy/ https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/ https://kubernetes.io/docs/concepts/services-networking/service/

This last page helped me a lot. If someone else gets to deploy SSL-termination and SSL-passthrough for either public or private connections, I hope this helps too.

Kubernetes internal nginx ingress controller with SSL termination & ssl-passthrough

2 Answers