How to customize the Security Group Ingress Rules created by a Kubernetes LoadBalancer type service that uses AWS NLB for TCP services

Question

I have a TCP service that runs on via a Kubernetes Deployment on an AWS EKS cluster and is exposed to the internet by a Service of type LoadBalancer using the following definition

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp
  name: tcpservice
spec:
  selector:
    app: tcpapp
  type: LoadBalancer
  ports:
  - port: 4453
    targetPort: 4453
    name: tcpport

Because the load balancer type is NLB, the ingress traffic has to be explicitly allowed on the security group that is applied to the nodes themselves. The security group was created like this:

✔ ~$ aws ec2 describe-security-groups --group-ids sg-2645567125762c6e2 | jq '.SecurityGroups[0].IpPermissions[0]'
{
  "FromPort": 32163,
  "IpProtocol": "tcp",
  "IpRanges": [
    {
      "CidrIp": "10.20.0.0/20",
      "Description": "kubernetes.io/rule/nlb/health=afd5427b6058811ea989512627425a2e"
    },
    {
      "CidrIp": "0.0.0.0/0",
      "Description": "kubernetes.io/rule/nlb/client=afd5427b6058811ea989512627425a2e"
    }
  ],
  "Ipv6Ranges": [],
  "PrefixListIds": [],
  "ToPort": 32163,
  "UserIdGroupPairs": []
}

So now I need to change the CidrIp in the "0.0.0.0/0" to a different block. How can I do this using kubernetes manifests? I've looked at the NetworkPolicy and Calico documentation, but this controls traffic to pods not services. I can change it with the AWS API or manually, but those changes are lost when the service is redeployed.

To confirm, you are trying to make any internal load balancer for something that isn’t in Kubernetes? Because for something else in k8s you wouldn’t use a load balancer type at all. — coderanger

iliefa iliefa · Accepted Answer · 2019-11-28T08:46:07

you need to add in your service manifest the loadBalancerSourceRanges parameter.

from documentation:

In order to limit which client IP’s can access the Network Load Balancer, specify loadBalancerSourceRanges.

spec:
  loadBalancerSourceRanges:
  - "143.231.0.0/16"

https://v1-13.docs.kubernetes.io/docs/concepts/services-networking/service/

how code is implemented can be found here:

https://github.com/kubernetes/kubernetes/blob/9d6ebf6c78f406d8639aae189901e47562418071/pkg/api/service/util.go

How to customize the Security Group Ingress Rules created by a Kubernetes LoadBalancer type service that uses AWS NLB for TCP services

1 Answers