How to setup Kubernetes NLB Load Balancer with target group “IP” based [AWS]?

Question

Currently, I'm exposing a k8s service using network load balancer. It creates a network load balancer and sets the target group as instance based and everything works fine. As we know port in the nodes is always in the range of 30000 - 32767.

There is a difference in the different target groups. Instance based target group is used to preserve the clientIP, where in IP based doesn't preserve the client IP.

Now there is a problem with the security group, I want to restrict the node ports only be accessible by the CIDR of load balancer. Since it is an instance based target group, inbound IP is always the client IP. So it is difficult to restrict the access only for certain IP's.

So my plan is to switch the target group to "IP" based, so that I can restrict the access to only for CIDR of load balancer.

Is there any other way to create the NLB load balancer with the IP based target type? Could you please help me with some suggestions?

apiVersion: v1
kind: Service
metadata:
  name: nginx-router
  annotations:
    service.beta.kubernetes.io/do-loadbalancer-protocol: "http"
    service.beta.kubernetes.io/do-loadbalancer-healthcheck-path: "/healthz"
    service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
    service.beta.kubernetes.io/aws-target-type: ip
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: [tes]

    # service.beta.kubernetes.io/healthcheck-path: /healthz
spec:
  selector:
    app: nginx-router
  externalTrafficPolicy: Local
  ports:
    - port: 80
      targetPort: 80
      protocol : TCP
      name : http
    - port : 443
      targetPort: 443
      protocol : TCP
      name : https

  type: LoadBalancer

So you want that your services are only reachable over the nginx-router (probably an ingress controller I guess)? So you don't want that your other services expose a node port? — BeWu
Yes that's correct. Currently that is working only when I give port access to 0.0.0.0/0 in the security group of nodes. I want to restrict the acess to port, so that should accessible only from loadbalancer. — Srikanth

BeWu BeWu · Accepted Answer · 2020-10-08T16:04:22

I ask myself if you really need to solve this through the Network Load Balancer or if a solution in Kubernetes would be preferable.

The easiest way to remove a NodePort from the cluster is to actually not define any Services in Kubernetes of the type NodePort. If some already exist you can easily change them to type ClusterIP and the NodePort should be removed.

Since you wish to prevent any access to NodePorts you can consider using a ResourceQuota to prevent the creation any services of type NodePort at all. This way the cluster is telling the user that his services won't work instead of just preventing the traffic from reaching the application and most likely resulting in a hard to understand timeout if you don't know the specifics of the load balancer configuration. (See here for reference: https://kubernetes.io/docs/concepts/policy/resource-quotas/#object-count-quota)

How to setup Kubernetes NLB Load Balancer with target group “IP” based [AWS]?

1 Answers