Kubernetes nginx ingress controller bad gateway

Question

I'm facing a strange issue in my K8S cluster

Basically I have 2 application:

identity manager (WSO2 IS based but the issue is not related to WSO2)
external SAML2 IDP that will manage X509 authentication

I configured WSO2 in order to use this external SAML2 IDP

When I try to login by X509, WSO2 shows me the login page, i click on smartcard and a redirect is done to the external SAML IDP.

In this case the nginx ingress gives to me 502 bad gateway. If I copy the URL, close the browser and try again to access directly to the X509 IDP, all works pretty good.

Note that I'm using another external SAML IDP and in this case the redirect is working pretty good

The difference between the 2 external IDP is that I configured the ingress controller of X509 SAML IDP in pass-through because I need that X509 certificare is consumed by my Java application

May, anybody, tell me why I'm having this strange behaviour?

Thank you

Angelo

UPDATE Here you can find my nginx.conf https://raw.githubusercontent.com/angeloimm/nginx_configuration/main/nginx.conf

This is my ingress.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: eid-tls-ingress
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
    nginx.ingress.kubernetes.io/rewrite-target: /eid-tsl/
    nginx.ingress.kubernetes.io/ssl-passthrough: "true"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
  tls:
    - hosts:
        - login-cns-test.it
  rules:
  - host: login-cns-test.it
    http:
      paths:
      - path: /
        backend:
          serviceName: eid-tls-service
          servicePort: 443

UPDATE 2

This is my scenario:

As you can see all http/s requests from internet are intercepted by my customer balancer (Balancer cloud vmware nsx) this balancer routes the requests to the worker nodes.

On worker nodes I have my eid-tls-service; it's a default type service (clusterIP type) so I need the ingress controller in order to handle request.

The only important thing (at least I think) is that I need a passthrough configuration. So I confogured my K8S and my nginx controller by using passthrough. No configuration has been done on Balancer cloud vmware nsx

In fact I need that the X509 certficate is not consumed by Ingress Controller but it must arrive directly to my application (to my service).

I have just 1 replica of my service.

This is my service yaml config:

kind: Service
apiVersion: v1
metadata:
  name: eid-tls-service
spec:
  selector:
    app: eid-tls
  ports:
  - protocol: TCP
    name: https-port
    port: 443
    targetPort: 443

From kubectl this is my eid-tls-service describe:

Name: eid-tls-service
Namespace: eid-tls-idp-ns
Labels: <none>
Annotations: Selector: app=eid-tls
Type: ClusterIP
IP: xx.ss.z.ttt
Port: https-port 443/TCP
TargetPort: 443/TCP
Endpoints: xx.yy.z.www:443
Session Affinity: None
Events: <none>

This is my ingress controller log error:

2021/01/28 11:24:06 [error] 3210#3210: *78115978 SSL_do_handshake() failed (SSL: error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate:SSL alert number 42) while SSL handshaking to upstream, client: 127.0.0.1, server:

What I really can't understand is why if I copy the URL, close the browser (by cleaning all cookies and files) and I paste the copied URL all works good (certificate is consumed by my java application)

Is it possible for you to share your ingress configuration with us ? — matt_j
Do the logs from the nginx-ingress controller give any hints? — Christian Fosli
@matt_j please see my update 2; did I answer to your question? — Angelo Immediata

Angelo Immediata Angelo Immediata · Accepted Answer · 2021-02-03T12:41:38

I guess I found the reason for this behaviour. Basically it's happening the following:

a HTTP request is handled by my IAM by using SSL connection
a redirect is made from my IAM to my X509 IAM living inside the saml K8S cluster.

My X509 IAM ingress controller is configured in pass-through. On step 2 the SSL connection is terminated and handled by my pod, K8S ingress controller tries to use SSL connection during the redirect and so the flow is compromised.

If I copy and paste the URL, I don't start a SSL connection on my IAM but I directly go to the X509 IAM so no redirect is done.

Basically I think I can't follow my approach so what I did is change the ingress.yaml definition in this way:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: eid-tls-ingress
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/configuration-snippet: "proxy_set_header X-SSL-CERT $ssl_client_escaped_cert;"
    nginx.ingress.kubernetes.io/server-snippet: "ssl_verify_client optional_no_ca;"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
  tls:
    - hosts:
        - login-cns-test.it
  rules:
  - host: login-cns-test.it
    http:
      paths:
      - path: /
        backend:
          serviceName: eid-tls-service
          servicePort: 443

I configured the ingress controller in way that the cetificate is passed to the backend application in a HTTP request header. Now it seems to be all pretty working.

Thank to all you

Kubernetes nginx ingress controller bad gateway

1 Answers