How to deploy ClusterRoleBinding in Google Kubernetes Engine for KubeIP

Question

I am seeing an RBAC failure when trying to deploy KubeIP to GKE.

I have isolated the issue down to the following section of the KubeIP infrastructure:

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: kubeip-sa
  namespace: kube-system
rules:
- apiGroups: [""]
  resources: ["nodes"]
  verbs: ["get","list","watch","patch"]
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get","list","watch"]

I am getting the following error from kubectl and GKE:

Error from server (Forbidden): error when creating "template.yml": clusterroles.rbac.authorization.k8s.io "kubeip-sa" is forbidden: attempt to grant extra privileges: [{[get] [] [nodes] [] []} {[list] [] [nodes] [] []} {[watch] [] [nodes] [] []} {[patch] [] [nodes] [] []} {[get] [] [pods] [] []} {[list] [] [pods] [] []} {[watch] [] [pods] [] []}] user=&{108986779198363313539 [system:authenticated] map[user-assertion.cloud.google.com:[AKUJVpldMDXqrDZ2slnJReDbLytxt6P2EEyEBbLNRB90oOATH4vIURo/lIhaBuAj9nnwwyxJDSxj2OdCyjjgBC/s5QxftIJnr8128ToTglCzk+e8Wybt4heIizRHugWnIhKNqkF+B0yiv0pIxgOfakma+SbkzbQbVzJPtgxsmHmak30YfPA58n/xyJ8R7oNVJ5dFUAWDFNsqHf/auolViw0Zd7Cr4aYYDXX4GScw==]]} ownerrules=[{[create] [authorization.k8s.io] [selfsubjectaccessreviews selfsubjectrulesreviews] [] []} {[get] [] [] [] [/api /api/* /apis /apis/* /healthz /openapi /openapi/* /swagger-2.0.0.pb-v1 /swagger.json /swaggerapi /swaggerapi/* /version /version/]}] ruleResolutionErrors=[]

I have crated the appropriate ~/.kube/config by issuing

gcloud container clusters get-credentials <cluster> \
  --zone <zone> \
  --project <project>

The gcloud service account I am using has been granted cluster-admin in the GKE cluster in question

kubectl create clusterrolebinding cluster-admin-binding \
  --clusterrole cluster-admin \
  --user $(gcloud config get-value account)

I can verify that my service account user should have the cluster-admin role checking my current gcloud user and checking the GKE ClusterRoleBinding

$ gcloud config get-value account
terraform@<project>.iam.gserviceaccount.com

$ kubectl describe clusterrolebinding cluster-admin-binding
Name:         cluster-admin-binding
Labels:       <none>
Annotations:  <none>
Role:
  Kind:  ClusterRole
  Name:  cluster-admin
Subjects:
  Kind  Name                                                Namespace
  ----  ----                                                ---------
  User  terraform@<project>.iam.gserviceaccount.com

According to kubectl I should be able to create ClusterRoleBindings

$ kubectl auth can-i create clusterrolebinding
yes

Does anybody see what element of GKE RBAC I am missing?

Nick Nick · Accepted Answer · 2019-03-28T17:29:45

The answer at this question "Creating a ClusterRole as the default compute service account fails with extra privileges error" guided me to the solution.

If you map the ClusterRoleBinding to the service account id instead of the email everything works as expected.

kubectl create clusterrolebinding cluster-admin-binding \
  --clusterrole cluster-admin \
  --user $(gcloud iam service-accounts describe <service account email> --format="value(uniqueId)")

How to deploy ClusterRoleBinding in Google Kubernetes Engine for KubeIP

1 Answers