terraform eks setup - namespaces is forbidden: User cannot list resource

Question

I have been following a guide on how to setup AWS EKS using terraform. https://learn.hashicorp.com/tutorials/terraform/eks

I am on the section where i need to authenticate the dashboard. https://learn.hashicorp.com/tutorials/terraform/eks#authenticate-the-dashboard

I have created the cluster roll binding

$ kubectl apply -f https://raw.githubusercontent.com/hashicorp/learn-terraform-provision-eks-cluster/master/kubernetes-dashboard-admin.rbac.yaml

I have generated the token

kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep service-controller-token | awk '{print $1}')

I have logged into the kubernetes dashboard using token. kubectl proxy

However after im logged in and i try to click on any of the panels to see the resources, i get a set of errors that are similar to the following.

namespaces is forbidden: User "system:serviceaccount:kube-system:service-controller" cannot list resource "namespaces" in API group "" at the cluster scope

cronjobs.batch is forbidden: User "system:serviceaccount:kube-system:service-controller" cannot list resource "cronjobs" in API group "batch" in the namespace "default"

The messages suggest to me the user im logged in as via the token does not have the permissions to view these resources. Although i am able to view them using kubectl cli tool.

kubectl describe clusterrole kubernetes-dashboard
Name:         kubernetes-dashboard
Labels:       k8s-app=kubernetes-dashboard
Annotations:  <none>
PolicyRule:
  Resources             Non-Resource URLs  Resource Names  Verbs
  ---------             -----------------  --------------  -----
  nodes.metrics.k8s.io  []                 []              [get list watch]
  pods.metrics.k8s.io   []                 []              [get list watch]

Which version of Kubernetes are you using 1.15? 1.16? 1.17? 1.19? — Meir Gabay

Meir Gabay Meir Gabay · Accepted Answer · 2021-03-27T11:05:41

The following will log you in as an admin-user, which seems to be the behavior you're looking for.

$ ADMIN_USER_TOKEN_NAME=$(kubectl -n kube-system get secret | grep admin-user-token | cut -d' ' -f1)
$ echo $ADMIN_USER_TOKEN_NAME

admin-user-token-k4s7r
# The suffix is auto-generated

$ ADMIN_USER_TOKEN_VALUE=$(kubectl -n kube-system get secret "$ADMIN_USER_TOKEN_NAME" -o jsonpath='{.data.token}' | base64 --decode)
$ echo "$ADMIN_USER_TOKEN_VALUE"

eyJhbGciOiJ ...
.....................-Tg
# Copy this token and use it on the Kubernetes Dashboard login page

The Service Account that was used in the tutorial is service-controller, which seems to have a very few permissions

$ kubectl -n kube-system describe clusterrole system:controller:service-controller
Name:         system:controller:service-controller
Labels:       kubernetes.io/bootstrapping=rbac-defaults
Annotations:  rbac.authorization.kubernetes.io/autoupdate: true
PolicyRule:
  Resources             Non-Resource URLs  Resource Names  Verbs
  ---------             -----------------  --------------  -----
  events                []                 []              [create patch update]
  events.events.k8s.io  []                 []              [create patch update]
  services              []                 []              [get list watch]
  nodes                 []                 []              [list watch]
  services/status       []                 []              [patch update]

Let me know if you have any issues.

terraform eks setup - namespaces is forbidden: User cannot list resource

1 Answers