Co locate pods and persistent volumes in kubernetes

1
votes

I have a kubernetes cluster spread across two zones, A and B. I am using nfs volumes for persistent storage. I have nfs volumes in both the zones. I am creating a stateful set of 2 replicas which will be spread across these zones (I used pod anti-affinity to achieve this). Now I want the pods in zone A to use the volumes in zone A and ones in zone B to use the volumes in zone B.

I can add labels to the persistent volumes and match the persistent volume claims with these labels. But how do I make sure that the pvc for a pod does not get bound to a pv in another zone?

1
kubernetesnfspersistent-volumespersistent-volume-claimskubernetes-cluster
How do you provision the PVs? If you're using dynamic volume provisioning with kubernetes.io/docs/concepts/storage/storage-classes and I'm not mistaken that should solve the problem? – Michael Hausenblas
There is no dynamic provisioning support for NFS as of now. And I am not sure if this -> github.com/kubernetes-incubator/external-storage is production ready. – raiyan
Yup, sorry, my bad. There's github.com/kubernetes-incubator/external-storage/tree/master/… but likely not production ready ;) – Michael Hausenblas
Is there any workaraound to achieve this? – raiyan
Hmmm, only thing I can think of right now is trying to see if you can use nodeAffinity (as described in kubernetes.io/docs/concepts/configuration/assign-pod-node) for it? – Michael Hausenblas

1 Answers

1
votes

You can try to bind persistent volume claims (PVCs) to persistent volumes (PVs) and split Kubernetes pods across your cluster between two zones using the special built-in label failure-domain.beta.kubernetes.io/zone. If you create volumes manually, it is possible to label them with failure-domain.beta.kubernetes.io/zone:zoneA value, ensuring that pod is only scheduled to nodes in the same zone as the zone of the persistent volume.

For example, to set label for a Node and PV:

kubectl label node <node-name> failure-domain.beta.kubernetes.io/zone=zoneA

kubectl label pv <pv-name> failure-domain.beta.kubernetes.io/zone=zoneA

Find some useful information from official Kubernetes documentation.