gke nginx ingress create additional load balancer

2
votes

I have a set of services that i want to expose as an ingress load balancer. I select nginx to be the ingress because of the ability to force http to https redirects.

Having an ingress config like 

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: api-https
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: true
    nginx.ingress.kubernetes.io/force-ssl-redirect: true
    nginx.org/ssl-services: "api,spa"
    kubernetes.io/ingress.class: nginx
spec:
  tls:
    - hosts:
        - api.some.com
        - www.some.com
      secretName: secret
  rules:
    - host: api.some.com
      http:
        paths:
          - path: /
            backend:
              serviceName: api
              servicePort: 8080
    - host: www.some.com
      http:
        paths:
          - path: /
            backend:
              serviceName: spa
              servicePort: 8081

gke creates the nginx ingress load balancer but also another load balancer with backends and everything like if where not nginx selected but gcp as ingress.

below screenshot shows in red the two unexpected LB and in blue the two nginx ingress LB one for our qa and prod env respectively.

gcplb

output from kubectl get services

xyz@cloudshell:~ (xyz)$ kubectl get services
NAME                            TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)                         AGE
api                             NodePort       1.2.3.4         <none>           8080:32332/TCP,4433:31866/TCP   10d
nginx-ingress-controller        LoadBalancer   1.2.6.9         12.13.14.15      80:32321/TCP,443:32514/TCP      2d
nginx-ingress-default-backend   ClusterIP      1.2.7.10        <none>           80/TCP                          2d
spa                             NodePort       1.2.8.11        <none>           8082:31847/TCP,4435:31116/TCP   6d

screenshot from gcp gke services view of the ingress with wrong info

ingress

Is this expected?

Did i miss any configuration to prevent this extra load balancer for been created?

2
nginxkuberneteskubernetes-ingressgoogle-kubernetes-enginenginx-ingress
Could you include some output to show what you mean about the additional load balancer? Where do you see it? – Ryan Dawson
added @RyanDawson – bitgandtter
Could you include output of 'kubectl get services' or 'kubectl get services --all-namespaces' if you have LoadBalancer services in another namespace. – Ryan Dawson
added @RyanDawson – bitgandtter

2 Answers

1
votes

On GCP GKE the gcp ingress controller its enabled by default and will be always lead to a new LB in any ingress definition even if the .class its specified.

https://github.com/kubernetes/ingress-nginx/issues/3703

So to fix it we should remove the gcp ingress controller from the cluster as mention on https://github.com/kubernetes/ingress-gce/blob/master/docs/faq/gce.md#how-do-i-disable-the-gce-ingress-controller

0
votes

When you create a deployment on GKE cluster, you have two possibilities to expose it:

  1. Use a Service with a type LoadBalancer and expose it - this will create a TCP load balancer
  2. Create a Service as a NodePort or a Cluster IP and expose it as an Ingress - this will create HTTP load balancer

If you can see both of them in Load Balancers, this means that you have probably created a Service type LoadBalancer and then exposed it as Ingress. You are opening the same deployment to be accessed from two different IPs, by service and Ingress. To confirm this try: 

$ kubectl get ingress
$ kubectl get svc

You will get 2 ips from these 2 commands and they will both show you the same page.

Better way to configure it is to have a service type NodePort, and expose that service as an ingress. This is especially useful because you can use the same ingress for exposing more services.

This way you are saving the number of IPs exposed (and saving money by not using several Load Balancers).