AKS | NetworkPolicy | Blocking ingress traffic while using Azure CNI

Question

I am having some trouble getting a basic NetworkPolicy resource to block all ingress traffic on an Azure Kubernetes Service (AKS) instance. AKS is set up with the azure network plugin (i.e., Azure CNI).

Our issue is that with VNet peering to an on-premises network, the AKS workloads are now exposed to bad actors from the internal network. So we have an ingress controller, but would like to make that the only entrypoint for all non-system workloads.

Here is the NetworkPolicy resource:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: hello-node-network-policy
  namespace: hello-namespace-2
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  ingress: []

On a Pod in a different namespace, I can still connect to both the Service endpoint and the Pod IP address (as visible in kubectl get pods --output=wide --namespace=hello-namespace-2). On an Azure VM in the same VNet, I am able to connect directly to the IP address as well.

The Namespace, StatefulSet, Service, Ingress, and NetworkPolicy definitions are below.

apiVersion: v1
kind: Namespace
metadata:
  name: hello-namespace-2
  labels:
    ingress-allowed: "allowed"
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  creationTimestamp: null
  labels:
    app: hello-node
  name: hello-node
  namespace: hello-namespace-2
spec:
  serviceName: hello-node
  replicas: 1
  selector:
    matchLabels:
      app: hello-node
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: hello-node
    spec:
      containers:
      - image: k8s.gcr.io/echoserver:1.4
        name: echoserver
        resources: {}
---
apiVersion: v1
kind: Service
metadata:
  name: hello-node-service
  namespace: hello-namespace-2
spec:
  type: ClusterIP
  selector:
    app: hello-node
  ports:
  - name: http
    protocol: TCP
    port: 80
    targetPort: 8080
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: hello-node-ingress
  namespace: hello-namespace-2
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
  rules:
  - host: hello-namespace-2
    http:
      paths:
      - path: /hello-node(/|$)(.*)
        backend:
          serviceName: hello-node-service
          servicePort: 80
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: hello-node-network-policy
  namespace: hello-namespace-2
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  ingress: []

This behaves like there is no network controller installed, which I thought Azure CNI's azure network plugin represented. Would we have to explicitly install a network controller like Calico?

Any insights into this behaviour is greatly appreciated.

Thanks!

Vit Vit · Accepted Answer · 2020-10-20T09:09:59

1. Network policy options in AKS

Azure provides two ways to implement network policy. You choose a network policy option when you create an AKS cluster. The policy option can't be changed after the cluster is created:

Azure's own implementation, called Azure Network Policies.
Calico Network Policies, an open-source network and network security solution founded by Tigera.

Both implementations use Linux IPTables to enforce the specified policies. Policies are translated into sets of allowed and disallowed IP pairs. These pairs are then programmed as IPTable filter rules.

2. Differences between Azure and Calico policies and their capabilities

3. Create an AKS cluster and enable network policy

To use Azure Network Policy, you must use the Azure CNI plug-in and define your own virtual network and subnets. For more detailed information on how to plan out the required subnet ranges, see configure advanced networking.

Calico Network Policy could be used with either this same Azure CNI plug-in or with the Kubenet CNI plug-in.

4. Personally me never used Azure CNI plug-in. Always created cluster using

az aks create --resource-group <RG> --name <NAME> --network-policy calico

Please take a look into examples:

a. Tutorial: Calico Network Policies with Azure Kubernetes Service

b. Network Policy in Kubernetes using Calico

AKS | NetworkPolicy | Blocking ingress traffic while using Azure CNI

1 Answers