Trying to set up an ingress with tls and open to some IPs only on GKE

Question

I'm having trouble setting up an ingress open only to some specific IPs, checked docs, tried a lot of stuff and an IP out of the source keep accessing. that's a Zabbix web interface on an alpine with nginx, set up a service on node-port 80 then used an ingress to set up a loadbalancer on GCP, it's all working, the web interface is working fine, but how can I make it accessible only to desired IPs? my firewall rules are ok and it's only accessible through load balancer IP

Also, I have a specific namespace for this deploy.

Cluster version 1.11.5-gke.5 EDIT i'm using GKE standard ingress GLBC

My template is config as follow can someone help enlighten me on what is missing:

    apiVersion: v1
    kind: ReplicationController
    metadata:
      name: zabbix-web
      namespace: zabbix-prod
      labels:
        app: zabbix
        tier: frontend
    spec:
      replicas: 1
      template:
        metadata:
          labels:
            name: zabbix-web
            app: zabbix
        spec:
          volumes:
          - name: cloudsql-instance-credentials
            secret:
              defaultMode: 420
              secretName: cloudsql-instance-credentials
          containers:
            - command:
              - /cloud_sql_proxy
              - -instances=<conection>
              - -credential_file=/secrets/cloudsql/credentials.json
              image: gcr.io/cloudsql-docker/gce-proxy:1.11
              imagePullPolicy: IfNotPresent
              name: cloudsql-proxy
              resources: {}
              securityContext:
                allowPrivilegeEscalation: false
                runAsUser: 2
              terminationMessagePath: /dev/termination-log
              terminationMessagePolicy: File
              volumeMounts:
              - mountPath: /secrets/cloudsql
                name: credentials
                readOnly: true
            - name: zabbix-web
              image: zabbix/zabbix-web-nginx-mysql:alpine-3.2-latest
              ports:
              - containerPort: 80
              env:
              - name: MYSQL_USER
                valueFrom:
                  secretKeyRef:
                    key: <user>
                    name: <user>
              - name: MYSQL_PASSWORD
                valueFrom:
                  secretKeyRef:
                    key: <pass>
                    name: <pass>
              - name: DB_SERVER_HOST
                value: 127.0.0.1
              - name: MYSQL_DATABASE
                value: <db>
              - name: ZBX_SERVER_HOST
                value: <db>
            readinessProbe:
              failureThreshold: 3
              httpGet:
                path: /index.php
                port: 80
                scheme: HTTP
              periodSeconds: 10
              successThreshold: 1
              timeoutSeconds: 30
---
apiVersion: v1
kind: Service
metadata:
  name: "zabbix-web-service"
  namespace: "zabbix-prod"
  labels:
    app: zabbix
spec:
  ports:
  - port: 80
    targetPort: 80
  selector:
    name: "zabbix-web"
  type: "NodePort"
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: zabbix-web-ingress
  namespace: zabbix-prod
  annotations:
    ingress.kubernetes.io/service.spec.externalTrafficPolicy: local
    ingress.kubernetes.io/whitelist-source-range: <xxx.xxx.xxx.xxx/32>
spec:
  tls:
  - secretName: <tls-cert>
  backend:
    serviceName: zabbix-web-service
    servicePort: 80

Erwin Rooijakkers Erwin Rooijakkers · Accepted Answer · 2020-07-01T11:25:02

You can whitelist IPs by configuring Ingress and Cloud Armour:

Switch to project:

gcloud config set project $PROJECT

Create a policy:

gcloud compute security-policies create $POLICY_NAME --description "whitelisting"

Change default policy to deny:

gcloud compute security-policies rules update 2147483647 --action=deny-403 \ 
  --security-policy $POLICY_NAME

On lower priority than the default whitelist all IPs you want to whitelist:

gcloud compute security-policies rules create 2 \
  --action allow \
  --security-policy $POLICY_NAME \
  --description "allow friends" \
  --src-ip-ranges "93.184.17.0/24,151.101.1.69/32"

With a maximum of ten per range.

Note you need valid CIDR ranges, for that you can use CIDR to IP Range -> IP Range to CIDR.

View the policy as follows:

gcloud compute security-policies describe $POLICY_NAME

To throw away an entry:

gcloud compute security-policies rules delete $PRIORITY --security-policy $POLICY_NAME

or the full policy:

gcloud compute security-policies delete $POLICY_NAME

Create a BackendConfig for the policy:

# File backendconfig.yaml:
apiVersion: cloud.google.com/v1
kind: BackendConfig
metadata:
  namespace: <namespace>
  name: <name>
spec:
  securityPolicy:
    name: $POLICY_NAME

$ kubectl apply -f backendconfig.yaml
backendconfig.cloud.google.com/backendconfig-name created

Add the BackendConfig to the Service:

metadata:
  namespace: <namespace>
  name: <service-name>
  labels:
    app: my-app
  annotations:
    cloud.google.com/backend-config: '{"ports": {"80":"backendconfig-name"}}'
spec:
  type: NodePort
  selector:
    app: hello-app
  ports:
  - port: 80
    protocol: TCP
    targetPort: 8080

Use the right selectors and point the receiving port of the Service to the BackendConfig created earlier.

Now Cloud Armour will add the policy to the GKE service.

Visible in https://console.cloud.google.com/net-security/securitypolicies (after selecting $PROJECT).

Trying to set up an ingress with tls and open to some IPs only on GKE

4 Answers