Issue with networkpolicies in kubernetes

Question

Create a namespace named forensics All the pods within the forensics namespace should not be able to communicate outside world (egress isolation) Create a pod named investigator in the default namespace. Pods within forensics namespace should only allow connection from IP of the investigator pod.

https://kubernetes.io/docs/concepts/services-networking/network-policies/

root@kubemaster:~/yaml# kubectl create namespace forensics --dry-run=client -o yaml > 03_ns-forensics.yaml
root@kubemaster:~/yaml# vi 03_ns-forensics.yaml

root@kubemaster:~/yaml# cat 03_ns-forensics.yaml 
apiVersion: v1
kind: Namespace
metadata:
  creationTimestamp: null
  name: forensics
  labels:
    name: forensics
spec: {}
status: {}

root@kubemaster:~/yaml# kubectl create -f 03_ns-forensics.yaml 
namespace/forensics created

root@kubemaster:~/yaml# kubectl get ns forensics --show-labels 
NAME        STATUS   AGE   LABELS
forensics   Active   31s   name=forensics

root@kubemaster:~/yaml# kubectl run test --image=busybox --image-pull-policy=IfNotPresent --namespace=forensics --command sleep --command 3600 --dry-run=client -o yaml > 03_pod-test.yaml

root@kubemaster:~/yaml# vi 03_pod-test.yaml

root@kubemaster:~/yaml# cat 03_pod-test.yaml
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: test
  name: test
  namespace: forensics
spec:
  containers:
  - command:
    - sleep
    - "3600"
    image: busybox
    imagePullPolicy: IfNotPresent
    name: test
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}


root@kubemaster:~/yaml# kubectl get pods -n forensics -o wide
NAME   READY   STATUS    RESTARTS   AGE   IP          NODE          NOMINATED NODE   READINESS GATES
test   1/1     Running   0          15s   10.36.0.3   kubeworker1   <none>           <none>


root@kubemaster:~/yaml# kubectl run investigator  --image=busybox --image-pull-policy=IfNotPresent  --command sleep --command 3600 --dry-run=client -o yaml > 03_pod-investigator.yaml


root@kubemaster:~/yaml# kubectl create -f 03_pod-investigator.yaml 
pod/investigator created

root@kubemaster:~/yaml# kubectl get pods investigator -o wide
NAME           READY   STATUS    RESTARTS   AGE   IP          NODE          NOMINATED NODE   READINESS GATES
investigator   1/1     Running   0          10s   10.44.0.5   kubeworker2   <none>           <none>


As per question, none of the pods from forensics should be able to communicate outside its namespace.

Before applying policy, pod from forensics can communicate to any other pod.

root@kubemaster:~/yaml# kubectl get pods investigator -o wide
NAME           READY   STATUS    RESTARTS   AGE    IP          NODE          NOMINATED NODE   READINESS GATES
investigator   1/1     Running   0          4m2s   10.44.0.5   kubeworker2   <none>           <none>
root@kubemaster:~/yaml# kubectl get pods -n forensics -o wide
NAME   READY   STATUS    RESTARTS   AGE     IP          NODE          NOMINATED NODE   READINESS GATES
test   1/1     Running   0          4m51s   10.36.0.3   kubeworker1   <none>           <none>


root@kubemaster:~/yaml# kubectl exec -it test -n forensics -- ping 10.44.0.5
PING 10.44.0.5 (10.44.0.5): 56 data bytes
64 bytes from 10.44.0.5: seq=0 ttl=64 time=9.726 ms
64 bytes from 10.44.0.5: seq=1 ttl=64 time=0.781 ms
^C
--- 10.44.0.5 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.781/5.253/9.726 ms

Create a default deny policy for egress from forensics namespace (applied to all pods)

root@kubemaster:~/yaml# cat 03_netpol-egress-forencis.yaml 
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-egress
  namespace: forensics
spec:
  podSelector: {}
  policyTypes:
  - Egress

root@kubemaster:~/yaml# kubectl create -f 03_netpol-egress-forencis.yaml 
networkpolicy.networking.k8s.io/default-deny-egress created
root@kubemaster:~/yaml# kubectl get networkpolicies.networking.k8s.io -n forensics 
NAME                  POD-SELECTOR   AGE
default-deny-egress   <none>         12s
root@kubemaster:~/yaml# kubectl describe networkpolicies.networking.k8s.io -n forensics 
Name:         default-deny-egress
Namespace:    forensics
Created on:   2020-05-08 05:56:14 +0000 UTC
Labels:       <none>
Annotations:  <none>
Spec:
  PodSelector:     <none> (Allowing the specific traffic to all pods in this namespace)
  Not affecting ingress traffic
  Allowing egress traffic:
    <none> (Selected pods are isolated for egress connectivity)
  Policy Types: Egress

Now it can not

root@kubemaster:~/yaml# kubectl exec -it test -n forensics -- ping 10.44.0.5
PING 10.44.0.5 (10.44.0.5): 56 data bytes
^C
--- 10.44.0.5 ping statistics ---
5 packets transmitted, 0 packets received, 100% packet loss
command terminated with exit code 1

Create another test pod in default name space for testing.

root@kubemaster:~/yaml# kubectl run test --image=busybox --image-pull-policy=IfNotPresent --command sleep --command 3600 --dry-run=client -o yaml > 03_pod-test-default-ns.yaml
root@kubemaster:~/yaml# kubectl create -f 03_pod-test-default-ns.yaml 
pod/test created

NAME   READY   STATUS    RESTARTS   AGE   IP          NODE          NOMINATED NODE   READINESS GATES
test   1/1     Running   0          17m   10.36.0.3   kubeworker1   <none>           <none>
root@kubemaster:~/yaml# kubectl exec -it test -- ping 10.36.0.3
PING 10.36.0.3 (10.36.0.3): 56 data bytes
64 bytes from 10.36.0.3: seq=0 ttl=64 time=8.701 ms
64 bytes from 10.36.0.3: seq=1 ttl=64 time=1.132 ms
^C
--- 10.36.0.3 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 1.132/4.916/8.701 ms

Currently other than investigator pod can still communicate with pods in forensic namespace.

Let’s apply policy to restricts traffic only from investigate pod only.

root@kubemaster:~/yaml# kubectl describe networkpolicies.networking.k8s.io network-policy-ingress -n forensics 
Name:         network-policy-ingress
Namespace:    forensics
Created on:   2020-05-08 06:51:42 +0000 UTC
Labels:       <none>
Annotations:  <none>
Spec:
  PodSelector:     <none> (Allowing the specific traffic to all pods in this namespace)
  Allowing ingress traffic:
    To Port: <any> (traffic allowed to all ports)
    From:
      PodSelector: run=investigator
  Not affecting egress traffic
  Policy Types: Ingress

root@kubemaster:~/yaml# kubectl get pods -n forensics -o wide
NAME   READY   STATUS    RESTARTS   AGE   IP          NODE          NOMINATED NODE   READINESS GATES
test   1/1     Running   1          63m   10.36.0.3   kubeworker1   <none>           <none>

root@kubemaster:~/yaml# kubectl get pods  -o wide
NAME                                             READY   STATUS    RESTARTS   AGE     IP          NODE          NOMINATED NODE   READINESS GATES
investigator                                     1/1     Running   1          63m     10.44.0.5   kubeworker2   <none>           <none>
kplabs-privileged                                1/1     Running   2          140m    10.36.0.2   kubeworker1   <none>           <none>
kplabs-secert-pod                                1/1     Running   3          3h12m   10.36.0.1   kubeworker1   <none>           <none>
nginx-ingress-controller-5bbc895c44-wzxq5        1/1     Running   2          2d      10.42.0.1   kubeworker3   <none>           <none>
nginx-ingress-default-backend-7c868597f4-xqcqn   1/1     Running   2          2d      10.44.0.2   kubeworker2   <none>           <none>
test                                             1/1     Running   0          48m     10.42.0.4   kubeworker3   <none>           <none>

root@kubemaster:~/yaml# kubectl exec -it investigator -- ping 10.36.0.3
PING 10.36.0.3 (10.36.0.3): 56 data bytes
^C
--- 10.36.0.3 ping statistics ---
5 packets transmitted, 0 packets received, 100% packet loss
command terminated with exit code 1
root@kubemaster:~/yaml#

What is wrong with network-policy-ingress ? its even not pinging from pod from default namespace.

I don't think this question meets the guidelines for the K8S tag you used. It should be about programming k8s to be on-topic. I've voted to move this to Server Fault, one of the other SO sites. I think you'll get more help there. — Software Engineer

shubham_asati shubham_asati · Accepted Answer · 2020-05-09T21:19:20

The problem is in your network policy. You should use something like this

- from
  - namespaceSelect:
      matchLabels:
        //select labels from default namespace
    podSelector:
      matchLabels:
        run: investigator

The problem with your network policy is that it will only allow Ingress from forensics namespace with podselector run: investigator. I guess this might help. For more details check this out https://kubernetes.io/docs/concepts/services-networking/network-policies/#behavior-of-to-and-from-selectors

Issue with networkpolicies in kubernetes

1 Answers