I am setting up my very first Kubernetes cluster. We are expecting to have mix of Windows and Linux node so I picked flannel as my cni. I am using RHEL 7.7 as my master node and I have two other RHEL 7.7 machines as worker node and then rest are Windows Server 2019. For most of the part, I was following documentation provided on Microsoft site: https://docs.microsoft.com/en-us/virtualization/windowscontainers/kubernetes/getting-started-kubernetes-windows and also one on Kubernetes site: https://kubernetes.cn/docs/tasks/administer-cluster/kubeadm/adding-windows-nodes/ . I know article on Microsoft site is more than 2 years old but this is only the guide I found for mixed mode operations.
I have done following so far on Master and worker RHEL nodes:
- stopped and disabled firewalld
- disabled selinux
- update && upgrade
- Disabled swap partition
- Added /etc/hosts entry for all nodes involved in my Kubernetes cluster
- Installed Docker CE 19.03.11
- Install kubectl, kubeadm and kubelet 1.18.3 (Build date 2020-05-20)
- Prepare Kubernetes control plane for Flannel:
sudo sysctl net.bridge.bridge-nf-call-iptables=1
I have now done following on RHEL Master node
Initialize cluster
kubeadm init --pod-network-cidr=10.244.0.0/16 --service-cidr=10.96.0.0/12
kubectl as non-root user
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Patch the Daemon set for the node selector
wget https://raw.githubusercontent.com/Microsoft/SDN/master/Kubernetes/flannel/l2bridge/manifests/node-selector-patch.yml
kubectl patch ds/kube-proxy --patch "$(cat node-selector-patch.yml)" -n=kube-system
After the patch, kube-proxy looks like this:
Add Flannel
wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
Modify the net-conf.json section of the flannel manifest in order to set the VNI to 4096 and the Port to 4789. It should look as follows:
net-conf.json: |
{
"Network": "10.244.0.0/16",
"Backend": {
"Type": "vxlan",
"VNI" : 4096,
"Port": 4789
}
}
Apply modified kube-flannel
kubectl apply -f kube-flannel.yml
After adding network, here is what I get for pods in kube-system
Add Windows Flannel and kube-proxy DaemonSets
curl -L https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/kube-proxy.yml | sed 's/VERSION/v1.18.0/g' | kubectl apply -f -
kubectl apply -f https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/flannel-overlay.yml
Join Worker node
I am now trying to join the RHEL 7.7 worker node by executing the kubeadm join command generated when IU initialized my cluster.
Worker node initializes fine as seen below:
when I go to my RHEL worker node, I see that k8s_install-cni_kube-flannel-ds-amd64-f4mtp_kube-system container is exited as seen below:
- Can you please let me know if I am following the correct procedure? I believe Flannel CNI is required to talk to pods within kubernetes cluster
- If Flannel is difficult to setup for mixed mode, can we use other network which can work?
- If we decide to go only and only RHEL nodes, what is the best and easiest network plugin I can install without going through lot of issues?
Thanks and I appreciate it.
