weave cni is not assigning ip address to weave bridge with in the node cidr in k8s

Question

I'm trying to bootstrap the k8s cluster using weave as cni plugin which was originally configured without --pod-cidr and weave plugin

root@kube1:/etc/systemd/system/kubelet.service.d# kubectl get no
NAME    STATUS   ROLES    AGE   VERSION
kube1   Ready    master   31m   v1.18.2
kube2   Ready    <none>   30m   v1.18.2
kube3   Ready    <none>   31m   v1.18.2
root@kube1:/etc/systemd/system/kubelet.service.d#

So i have done the cleanup using the below commands

 kubectl drain kube2 --delete-local-data --force --ignore-daemonsets
 kubectl drain kube3 --delete-local-data --force --ignore-daemonsets
 kubectl drain kube1 --delete-local-data --force --ignore-daemonsets

 kubectl delete no kube1 kube2 kube3

 kubeadm reset

 curl -L git.io/weave -o /usr/local/bin/weave
 chmod a+x /usr/local/bin/weave
 kubeadm reset
 weave reset --force

 rm /opt/cni/bin/weave-*

 iptables -P INPUT ACCEPT
 iptables -P FORWARD ACCEPT
 iptables -P OUTPUT ACCEPT
 iptables -t nat -F
 iptables -t mangle -F
 iptables -F
 iptables -X
 systemctl restart docker

i have ensured the weave bridge interface was deleted on all 3 nodes and re initiated the cluster

  kubeadm init --apiserver-advertise-address=192.168.56.101 --pod-network-cidr=10.244.0.0/16 --service-cidr=10.100.0.0/16

I have ensured the node-cidr was allocated to worker nodes(pod-cidr - 10.244.0.0./16) as below

root@kube1:/etc/systemd/system/kubelet.service.d# kubectl get no kube2 -o yaml|grep -i podCIDR|grep -i 24
  podCIDR: 10.244.2.0/24

root@kube1:/etc/systemd/system/kubelet.service.d# kubectl get no kube3 -o yaml|grep -i podCIDR|grep -i 24
  podCIDR: 10.244.1.0/24

After i have created the weave pod's ,i was expecting to see the weave bridge interface ip in 10.244.* but it created with default weave configuration (10.32.0.1) it seems

root@kube2:/etc/kubernetes# ifconfig weave
weave: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1376
        inet 10.32.0.1  netmask 255.240.0.0  broadcast 10.47.255.255
        inet6 fe80::187b:63ff:fe5c:a2ae  prefixlen 64  scopeid 0x20<link>
        ether 1a:7b:63:5c:a2:ae  txqueuelen 1000  (Ethernet)

Is there anything i have missed to clean up? or is it weave plugins default behaviour?

Bryan Bryan · Accepted Answer · 2020-05-22T12:09:30

By default Weave Net uses its own IP allocator, which can be configured via the environment variable IPALLOC_RANGE. Link to docs

If you change the CNI config on each node to use a different IPAM plugin, e.g. "host-local" you can probably get that to do exactly what you tried to do.

weave cni is not assigning ip address to weave bridge with in the node cidr in k8s

1 Answers