I try to made a first experience on kubernetes by practice.
kubernetes v1.9 has been setup on 5 raspberry pi mounted as cluster.
OS : hypriot v1.4
host / static ip configured / raspberry hardware version :
- master: 192.168.1.230 / rpi v3
- node01: 192.168.1.231 / rpi v3
- node02: 192.168.1.232 / rpi v3
- node03: 192.168.1.233 / rpi v2
- node04: 192.168.1.234 / rpi v2
For the pod network I choose Weave Net. Traefik has been installed in the node01 as load balancer to access my service from outside. I ssh the master and use these commands to install it (origin: https://blog.hypriot.com/post/setup-kubernetes-raspberry-pi-cluster/) :
$ kubectl apply -f https://raw.githubusercontent.com/hypriot/rpi-traefik/master/traefik-k8s-example.yaml
$ kubectl label node node01 nginx-controller=traefik
All system pods are running.
$ kubectl get pods --all-namespaces
kube-system etcd-master 1/1 Running 5 22h
kube-system kube-apiserver-master 1/1 Running 40 13h
kube-system kube-controller-manager-master 1/1 Running 10 13h
kube-system kube-dns-7b6ff86f69-x58pj 3/3 Running 9 23h
kube-system kube-proxy-5bqwh 1/1 Running 2 15h
kube-system kube-proxy-kngp9 1/1 Running 2 16h
kube-system kube-proxy-n85xl 1/1 Running 5 23h
kube-system kube-proxy-ncg2k 1/1 Running 2 15h
kube-system kube-proxy-qbfcf 1/1 Running 2 21h
kube-system kube-scheduler-master 1/1 Running 5 22h
kube-system traefik-ingress-controller-9dc7454cc-7rhpf 1/1 Running 1 14h
kube-system weave-net-6mvc6 2/2 Running 31 15h
kube-system weave-net-8hff9 2/2 Running 31 15h
kube-system weave-net-9kwgr 2/2 Running 31 21h
kube-system weave-net-llgrk 2/2 Running 41 22h
kube-system weave-net-s2h62 2/2 Running 29 16h
The issue is when I try to connect to the node01 by using this url http://192.168.1.231/. I got a 404 page not found...
So I checked the log and figure out that they are a problem with the default account :
$ kubectl logs traefik-ingress-controller-9dc7454cc-7rhpf
ERROR: logging before flag.Parse: E1226 07:29:15.195193 1 reflector.go:199] github.com/containous/traefik/vendor/k8s.io/client-go/tools/cache/reflector.go:94: Failed to list *v1.Endpoints: endpoints is forbidden: User "system:serviceaccount:kube-system:default" cannot list endpoints at the cluster scope
ERROR: logging before flag.Parse: E1226 07:29:15.422807 1 reflector.go:199] github.com/containous/traefik/vendor/k8s.io/client-go/tools/cache/reflector.go:94: Failed to list *v1.Secret: secrets is forbidden: User "system:serviceaccount:kube-system:default" cannot list secrets at the cluster scope
ERROR: logging before flag.Parse: E1226 07:29:15.915317 1 reflector.go:199] github.com/containous/traefik/vendor/k8s.io/client-go/tools/cache/reflector.go:94: Failed to list *v1.Service: services is forbidden: User "system:serviceaccount:kube-system:default" cannot list services at the cluster scope
ERROR: logging before flag.Parse: E1226 07:29:16.108385 1 reflector.go:199] github.com/containous/traefik/vendor/k8s.io/client-go/tools/cache/reflector.go:94: Failed to list *v1beta1.Ingress: ingresses.extensions is forbidden: User "system:serviceaccount:kube-system:default" cannot list ingresses.extensions at the cluster scope
Is it really a problem with the account system:serviceaccount:kube-system:default used? What account should I use instead of?
Thanks for helping.
Additional informations:
$ docker -v Docker version 17.03.0-ce, build 60ccb22
$ kubectl describe pods traefik-ingress-controller -n kube-system
Name: traefik-ingress-controller-9dc7454cc-7rhpf
Namespace: kube-system
Node: node01/192.168.1.231
Start Time: Mon, 25 Dec 2017 20:54:45 +0000
Labels: k8s-app=traefik-ingress-controller
pod-template-hash=587301077
Annotations: scheduler.alpha.kubernetes.io/tolerations=[
{
"key": "dedicated",
"operator": "Equal",
"value": "master",
"effect": "NoSchedule"
}
]
Status: Running
IP: 192.168.1.231
Controlled By: ReplicaSet/traefik-ingress-controller-9dc7454cc
Containers:
traefik-ingress-controller:
Container ID: docker://9e28800da6937a48aa20b5ef6526846b321a516ad20ee24ea3d32876f6769531
Image: hypriot/rpi-traefik
Image ID: docker-pullable://hypriot/rpi-traefik@sha256:ecdfcd94571ec8c121c20a6ec616d68aeaad93150a0717260196f813e31737d9
Ports: 80/TCP, 8888/TCP
Args:
--web
--web.address=localhost:8888
--kubernetes
State: Running
Started: Mon, 25 Dec 2017 22:24:33 +0000
Last State: Terminated
Reason: Error
Exit Code: 255
Started: Mon, 25 Dec 2017 20:54:50 +0000
Finished: Mon, 25 Dec 2017 22:17:09 +0000
Ready: True
Restart Count: 1
Limits:
cpu: 200m
memory: 30Mi
Requests:
cpu: 100m
memory: 20Mi
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-4wzhl (ro)
Conditions:
Type Status
Initialized True
Ready True
PodScheduled True
Volumes:
default-token-4wzhl:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-4wzhl
Optional: false
QoS Class: Burstable
Node-Selectors: nginx-controller=traefik
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events: <none>
Name: traefik-ingress-controller-9dc7454cc-jszgz
Namespace: kube-system
Node: node01/
Start Time: Mon, 25 Dec 2017 18:28:21 +0000
Labels: k8s-app=traefik-ingress-controller
pod-template-hash=587301077
Annotations: scheduler.alpha.kubernetes.io/tolerations=[
{
"key": "dedicated",
"operator": "Equal",
"value": "master",
"effect": "NoSchedule"
}
]
Status: Failed
Reason: MatchNodeSelector
Message: Pod Predicate MatchNodeSelector failed
IP:
Controlled By: ReplicaSet/traefik-ingress-controller-9dc7454cc
Containers:
traefik-ingress-controller:
Image: hypriot/rpi-traefik
Ports: 80/TCP, 8888/TCP
Args:
--web
--web.address=localhost:8888
--kubernetes
Limits:
cpu: 200m
memory: 30Mi
Requests:
cpu: 100m
memory: 20Mi
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-4wzhl (ro)
Volumes:
default-token-4wzhl:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-4wzhl
Optional: false
QoS Class: Burstable
Node-Selectors: nginx-controller=traefik
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events: <none>
$ kubectl describe pods weave-net-9kwgr -n kube-system
Name: weave-net-llgrk
Namespace: kube-system
Node: master/192.168.1.230
Start Time: Mon, 25 Dec 2017 13:33:40 +0000
Labels: controller-revision-hash=2209123374
name=weave-net
pod-template-generation=1
Annotations: <none>
Status: Running
IP: 192.168.1.230
Controlled By: DaemonSet/weave-net
Containers:
weave:
Container ID: docker://7824b8b02f1a8f5a53d7f27f0c12b44f73a4b666a694b974142f974294bedd6c
Image: weaveworks/weave-kube:2.1.3
Image ID: docker-pullable://weaveworks/weave-kube@sha256:07a3d56b8592ea3e00ace6f2c3eb7e65f3cc4945188a9e2a884b8172e6a0007e
Port: <none>
Command:
/home/weave/launch.sh
State: Running
Started: Tue, 26 Dec 2017 00:13:58 +0000
Last State: Terminated
Reason: Error
Exit Code: 1
Started: Tue, 26 Dec 2017 00:08:38 +0000
Finished: Tue, 26 Dec 2017 00:08:50 +0000
Ready: True
Restart Count: 37
Requests:
cpu: 10m
Liveness: http-get http://127.0.0.1:6784/status delay=30s timeout=1s period=10s #success=1 #failure=3
Environment:
HOSTNAME: (v1:spec.nodeName)
Mounts:
/host/etc from cni-conf (rw)
/host/home from cni-bin2 (rw)
/host/opt from cni-bin (rw)
/host/var/lib/dbus from dbus (rw)
/lib/modules from lib-modules (rw)
/run/xtables.lock from xtables-lock (rw)
/var/run/secrets/kubernetes.io/serviceaccount from weave-net-token-mx5jk (ro)
/weavedb from weavedb (rw)
weave-npc:
Container ID: docker://b199904c10ed34501748c25e13862113aeb32c7779b0797d72c95f9e9d868331
Image: weaveworks/weave-npc:2.1.3
Image ID: docker-pullable://weaveworks/weave-npc@sha256:f35eb8166d7dae3fa7bb4d9892ab6dc8ea5c969f73791be590a0a213767c0f07
Port: <none>
State: Running
Started: Mon, 25 Dec 2017 22:24:32 +0000
Last State: Terminated
Reason: Error
Exit Code: 255
Started: Mon, 25 Dec 2017 20:54:30 +0000
Finished: Mon, 25 Dec 2017 22:17:09 +0000
Ready: True
Restart Count: 4
Requests:
cpu: 10m
Environment:
HOSTNAME: (v1:spec.nodeName)
Mounts:
/run/xtables.lock from xtables-lock (rw)
/var/run/secrets/kubernetes.io/serviceaccount from weave-net-token-mx5jk (ro)
Conditions:
Type Status
Initialized True
Ready True
PodScheduled True
Volumes:
weavedb:
Type: HostPath (bare host directory volume)
Path: /var/lib/weave
HostPathType:
cni-bin:
Type: HostPath (bare host directory volume)
Path: /opt
HostPathType:
cni-bin2:
Type: HostPath (bare host directory volume)
Path: /home
HostPathType:
cni-conf:
Type: HostPath (bare host directory volume)
Path: /etc
HostPathType:
dbus:
Type: HostPath (bare host directory volume)
Path: /var/lib/dbus
HostPathType:
lib-modules:
Type: HostPath (bare host directory volume)
Path: /lib/modules
HostPathType:
xtables-lock:
Type: HostPath (bare host directory volume)
Path: /run/xtables.lock
HostPathType:
weave-net-token-mx5jk:
Type: Secret (a volume populated by a Secret)
SecretName: weave-net-token-mx5jk
Optional: false
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: :NoSchedule
node.kubernetes.io/disk-pressure:NoSchedule
node.kubernetes.io/memory-pressure:NoSchedule
node.kubernetes.io/not-ready:NoExecute
node.kubernetes.io/unreachable:NoExecute
Events: <none>
root@master:/home/pirate# kubectl describe pods weave-net-9kwgr -n kube-system
Name: weave-net-9kwgr
Namespace: kube-system
Node: node01/192.168.1.231
Start Time: Mon, 25 Dec 2017 14:50:37 +0000
Labels: controller-revision-hash=2209123374
name=weave-net
pod-template-generation=1
Annotations: <none>
Status: Running
IP: 192.168.1.231
Controlled By: DaemonSet/weave-net
Containers:
weave:
Container ID: docker://92e31f645b4dcd41e4d8189a6f67fa70a395971e071d635dc4c4208b8d1daf63
Image: weaveworks/weave-kube:2.1.3
Image ID: docker-pullable://weaveworks/weave-kube@sha256:07a3d56b8592ea3e00ace6f2c3eb7e65f3cc4945188a9e2a884b8172e6a0007e
Port: <none>
Command:
/home/weave/launch.sh
State: Running
Started: Tue, 26 Dec 2017 00:13:39 +0000
Last State: Terminated
Reason: Error
Exit Code: 1
Started: Tue, 26 Dec 2017 00:08:17 +0000
Finished: Tue, 26 Dec 2017 00:08:28 +0000
Ready: True
Restart Count: 29
Requests:
cpu: 10m
Liveness: http-get http://127.0.0.1:6784/status delay=30s timeout=1s period=10s #success=1 #failure=3
Environment:
HOSTNAME: (v1:spec.nodeName)
Mounts:
/host/etc from cni-conf (rw)
/host/home from cni-bin2 (rw)
/host/opt from cni-bin (rw)
/host/var/lib/dbus from dbus (rw)
/lib/modules from lib-modules (rw)
/run/xtables.lock from xtables-lock (rw)
/var/run/secrets/kubernetes.io/serviceaccount from weave-net-token-mx5jk (ro)
/weavedb from weavedb (rw)
weave-npc:
Container ID: docker://ddd86bef74d3fd40134c8609551cc07658aa62a2ede7ce51aec394001049e96d
Image: weaveworks/weave-npc:2.1.3
Image ID: docker-pullable://weaveworks/weave-npc@sha256:f35eb8166d7dae3fa7bb4d9892ab6dc8ea5c969f73791be590a0a213767c0f07
Port: <none>
State: Running
Started: Mon, 25 Dec 2017 22:24:32 +0000
Last State: Terminated
Reason: Error
Exit Code: 255
Started: Mon, 25 Dec 2017 20:54:30 +0000
Finished: Mon, 25 Dec 2017 22:17:09 +0000
Ready: True
Restart Count: 2
Requests:
cpu: 10m
Environment:
HOSTNAME: (v1:spec.nodeName)
Mounts:
/run/xtables.lock from xtables-lock (rw)
/var/run/secrets/kubernetes.io/serviceaccount from weave-net-token-mx5jk (ro)
Conditions:
Type Status
Initialized True
Ready True
PodScheduled True
Volumes:
weavedb:
Type: HostPath (bare host directory volume)
Path: /var/lib/weave
HostPathType:
cni-bin:
Type: HostPath (bare host directory volume)
Path: /opt
HostPathType:
cni-bin2:
Type: HostPath (bare host directory volume)
Path: /home
HostPathType:
cni-conf:
Type: HostPath (bare host directory volume)
Path: /etc
HostPathType:
dbus:
Type: HostPath (bare host directory volume)
Path: /var/lib/dbus
HostPathType:
lib-modules:
Type: HostPath (bare host directory volume)
Path: /lib/modules
HostPathType:
xtables-lock:
Type: HostPath (bare host directory volume)
Path: /run/xtables.lock
HostPathType:
weave-net-token-mx5jk:
Type: Secret (a volume populated by a Secret)
SecretName: weave-net-token-mx5jk
Optional: false
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: :NoSchedule
node.kubernetes.io/disk-pressure:NoSchedule
node.kubernetes.io/memory-pressure:NoSchedule
node.kubernetes.io/not-ready:NoExecute
node.kubernetes.io/unreachable:NoExecute
Events: <none>
