using nfs network path as a kubernetes persistence volume

Question

I have setup a kubernetes cluster with three nodes. All nodes are Linux centos machines.

I need persistent volume to store data and I am trying to achive this.

I was following this tutorial. But it only covers a one node cluster. https://kubernetes.io/docs/tutorials/stateful-application/mysql-wordpress-persistent-volume/

Since, my cluster consist of three node, I could not use local path. Previous tutorial does not worked for me.

I need a network path and using NFS seems a reasonable solution to me. ( Is there any good alternative I would like to hear.)

Using NFS network mount contains two steps. First, Creating a persistent volume on a network path. Second, define this network path as a persistent volume and use it.

Second step pretty straight forward. Its is explained in kubernetes documentation and there is even sample yaml. documentation:https://kubernetes.io/docs/concepts/storage/persistent-volumes/#persistent-volumes example: https://github.com/kubernetes/examples/blob/master/staging/volumes/nfs/nfs-pv.yaml

First part also seems straight forward. Its explained in following document

https://www.digitalocean.com/community/tutorials/how-to-set-up-an-nfs-mount-on-ubuntu-16-04#step-5-%E2%80%94-creating-the-mount-points-on-the-client

/etc/exports
directory_to_share    client(share_option1,...,share_optionN)

/etc/exports
/var/nfs/general    203.0.113.256(rw,sync,no_subtree_check)
/home       203.0.113.256(rw,sync,no_root_squash,no_subtree_check)

But when you export a path as a NFS path you should make some configuration and give clients some rights. Basically you need client ip.

With kubernetes we use abstraction such as pods and we don't want to deal with real machines and theirs ip addresses. So, the problem startes here.

So, I don't want to give nodes ip to nfs server. (They might change in he first place.) There should be a better solution that all pods (in any node) should be able to connect to NFS network path.

Even allowing all ip without restriction or allowing an ip range might solve the issue. I would like to hear if there is such solution. But, I would also like to hear what is the best practice. How everybody else use NFS network path from kubernetes.

I could not find any solution yet. If you solved similar problem, please let me know how you solve it. Any documenatation on this issue will be good too. Thanks in advance!

I don't want to give nodes ip to nfs server but yet you are cool hard-coding whomever owns 203.0.113.256? Because I would expect those addresses to change with the same frequency. Sure, if you are running the cluster-autoscaler then new Nodes might just magically appear, but I would say that's not the typical case — mdaniel
@MatthewLDaniel, I am not experienced kubernetes networking! Should I give node ip or pod ip to the nfs server. When a pod reach the nfs server, what will be the request source ip, node ip, pod ip or something else.Since NFS server outside the kubernetes cluster, the request will go on real network and I assume it should be node ip. But, I am not sure, any information would be very helpful. — clockworks
I am not sure whether it is a best practice or not but there is a possible solution here: stackoverflow.com/questions/45360084/… — clockworks
Yes, it will be the Node's IP, because it is the underlying machine that will establish the NFS mount (I'm not sure what you mean by "source ip"); by the time the Pod is running, that has already taken place and is just volume-mounted into the container — mdaniel

aurelius aurelius · Accepted Answer · 2018-06-29T13:14:41

You asked for best practices and from what I've found I think that the best option would be white-listing the IP addresses. Since you do not want to do that, there are also some workarounds answers posted on SO created by people who had similar issues with dynamic IP clients in NFS. You can find a link to deployment using glusterfs in the posted answers. If you want NFS with dynamic IP (because it can change), you can use DNS names instead of IP. If you need dynamic provisioning, use glusterfs.

I will add some information about the volumes as you asked. Might give you some light on the topic and help with the issue.
Since pods are ephemeral you need to move the volume outside the Pod - therefore making it independent from the Pods - so the volume would persist its state in case of Pod failure. Kubernetes supports several types of Volumes.
You could use NFS (more on NFS in the previous link) - so after the Pod is removed the volume gets unmounted, but it still exists. This is also not desired in your situation as the user needs to know the file system type and other details about the volume it will want to connect to. When you go to the examples in the documentation about NFS yaml files, you will see that their kind is defined as a Persistent Volume Claim.
This method is based on creating a series of abstractions that will allow a Node to connect to the Persistent Volume, but the user will not need any backend details, in addition, your node can connect to many providers.

using nfs network path as a kubernetes persistence volume

1 Answers