How to make websocket working in kubernetes ingress?

Question

I am dockering our current application and deploying on kubernetes cluster. We have 2 services, namely, service-A and service-B. One of our services(example service-A) uses websocket. we have configured a rule in ingress to route the websocket request directly to service-A on port 8080. Also have a rule to route other requests to service-B on port 443. But ingress controller always route the websocket request to service-B instead of routing to service-A.

So, I removed the service-B rule from ingress, but still its routed as tls request and request never reaches service-A. Not sure why its rerouted as TLS instead of http request upgraded to websocket connection.

Please find my ingress configuration below:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    field.cattle.io/publicEndpoints: '[{"addresses":[""],"port":443,"protocol":"HTTPS","serviceName":"cluster42:service-B","ingressName":"cluster42:my-ingress","hostname":"cluster42-phase-0 ","path":"/","allNodes":false},{"addresses":[""],"port":443,"protocol":"HTTPS","serviceName":"cluster42:service-A","ingressName":"cluster42:my-ingress","hostname":"cluster42-phase-0 ","path":"/ws-service","allNodes":false}]'
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"nginx.ingress.kubernetes.io/rewrite-target":"/","nginx.ingress.kubernetes.io/ssl-passthrough":"true"},"labels":{"app":"ingress","chart":" myapplication-chart","heritage":"Tiller","release":"installation-cluster42"},"name":"my-ingress","namespace":"cluster42"},"spec":{"rules":[{"host":"cluster42-phase-0 ","http":{"paths":[{"backend":{"serviceName":"service-B","servicePort":443},"path":"/"}]}}],"tls":[{"hosts":["cluster42-phase-0 "]}]}}
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/ssl-passthrough: "true"
    nginx.org/websocket-services: "service-A"
  creationTimestamp: "2019-05-24T11:46:40Z"
  generation: 31
  labels:
    app: ingress
    chart:  myapplication-chart
    heritage: Tiller
    release: installation-cluster42
  name: my-ingress
  namespace: cluster42
  resourceVersion: "57549362"
  selfLink: /apis/extensions/v1beta1/namespaces/cluster42/ingresses/my-ingress
  uid: 98784b1f-7e19-11e9-b2f1-005056b0b58e
spec:
  rules:
  - host: cluster42-phase-0 
    http:
      paths:
      - backend:
          serviceName: service-B
          servicePort: 443
        path: /
      - backend:
          serviceName: service-A
          servicePort: 8080
        path: /ws-service
  tls:
  - hosts:
    - cluster42-phase-0 
status:
  loadBalancer:
    ingress:
    - {}

I expect the request to be routed to service-A instead of service-B. Can you please let me know if I am missing something in my configuration or doing anything wrong.

Thanks in Advance.

Malgorzata Malgorzata · Accepted Answer · 2019-06-26T07:51:38

Check if destination port are opened and not used.

Then check if you have enough right to access by tiller to kube-system namespace. otherwise you have to create RBAC and special service. More infromation you can find here: tiller-rbac.

You have some mistakes in your ingress configuration file in spec section.

Take notice that currently the Ingress only supports a single TLS port, 443, and assumes TLS termination. So it's obvious that destination service will be service B with port 443. So you can delete tls section from configuration file.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    field.cattle.io/publicEndpoints: '[{"addresses":[""],"port":443,"protocol":"HTTPS","serviceName":"cluster42:service-B","ingressName":"cluster42:my-ingress","hostname":"cluster42-phase-0 ","path":"/","allNodes":false},{"addresses":[""],"port":443,"protocol":"HTTPS","serviceName":"cluster42:service-A","ingressName":"cluster42:my-ingress","hostname":"cluster42-phase-0 ","path":"/ws-service","allNodes":false}]'
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"nginx.ingress.kubernetes.io/rewrite-target":"/","nginx.ingress.kubernetes.io/ssl-passthrough":"true"},"labels":{"app":"ingress","chart":" myapplication-chart","heritage":"Tiller","release":"installation-cluster42"},"name":"my-ingress","namespace":"cluster42"},"spec":{"rules":[{"host":"cluster42-phase-0 ","http":{"paths":[{"backend":{"serviceName":"service-B","servicePort":443},"path":"/"}]}}],"tls":[{"hosts":["cluster42-phase-0 "]}]}}
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/ssl-passthrough: "true"
    nginx.org/websocket-services: "service-A"
  creationTimestamp: "2019-05-24T11:46:40Z"
  generation: 31
  labels:
    app: ingress
    chart:  myapplication-chart
    heritage: Tiller
    release: installation-cluster42
  name: my-ingress
  namespace: cluster42
  resourceVersion: "57549362"
  selfLink: /apis/extensions/v1beta1/namespaces/cluster42/ingresses/my-ingress
  uid: 98784b1f-7e19-11e9-b2f1-005056b0b58e
spec:
  rules:
  - host: cluster42-phase-0 
    http:
      paths:
      - path: /
        backend:
          serviceName: service-B
          servicePort: 443
      - path: /ws-service
        backend:
          serviceName: service-A
          servicePort: 8080
status:
  loadBalancer:
    ingress:
    - {}

How to make websocket working in kubernetes ingress?

1 Answers