Accessing Kubernetes API in GKE using service accounts

Question

I have my Kubernetes cluster running in GKE I want to run an application outside the cluster and talk to the Kubernetes API.

By using password retrieved from running:

gcloud container clusters get-credentials cluster-2 --log-http

I am able to access the API with Basic authentication. But I want to create multiple Kubernetes service accounts and configure them with required authorization and use appropriately.

So, I created service accounts and obtained the tokens using:

kubectl create serviceaccount sauser1
kubectl get serviceaccounts sauser1 -o yaml
kubectl get secret sauser1-token-<random-string-as-retrieved-from-previous-command> -o yaml

If I try to access the Kubernetes API with the obtained token using Bearer authentication then I get a 401 HTTP error. I thought that some permissions may have to be set for the service account, so based on the documentation here, I created below YAML file:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: pod-reader
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "watch", "list"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: pod-reader
subjects:
- kind: ServiceAccount
  name: sauser1
  namespace: default
roleRef:
  kind: ClusterRole
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

and tried to apply it using the below command:

kubectl apply -f default-sa-rolebinding.yaml

I got the following error:

clusterrolebinding "pod-reader" created
Error from server (Forbidden): error when creating "default-sa-rolebinding.yaml"
: clusterroles.rbac.authorization.k8s.io "pod-reader" is forbidden: attempt to g
rant extra privileges: [PolicyRule{Resources:["pods"], APIGroups:[""], Verbs:["g
et"]} PolicyRule{Resources:["pods"], APIGroups:[""], Verbs:["watch"]} PolicyRule
{Resources:["pods"], APIGroups:[""], Verbs:["list"]}] user=&{xyz@gmail.
com  [system:authenticated] map[authenticator:[GKE]]} ownerrules=[PolicyRule{Res
ources:["selfsubjectaccessreviews"], APIGroups:["authorization.k8s.io"], Verbs:[
"create"]} PolicyRule{NonResourceURLs:["/api" "/api/*" "/apis" "/apis/*" "/healt
hz" "/swagger-2.0.0.pb-v1" "/swagger.json" "/swaggerapi" "/swaggerapi/*" "/versi
on"], Verbs:["get"]}] ruleResolutionErrors=[]

I dont know how to proceed from here. Is my approach correct or am I missing something here?

UPDATE: As per the post referred by @JanosLenart in the comments, modified the kubectl command and the above error is not observed. But accessing the API, still gives 401 error. The curl command that I am using is:

curl -k -1 -H "Authorization: Bearer <token>" https://<ip-address>/api/v1/namespaces/default/pods -v

Possible duplicate of GKE clusterrolebinding for cluster-admin fails with permission error. Your approach is sound otherwise! — Janos Lenart
@JanosLenart The other post, resolved the forbidden error that I got when creating clusterrole. But accessing the API, still gives 401 error. The curl command that I am using is: curl -k -1 -H "Authorization: Bearer <token>" https://<ip-address>/api/v1/namespaces/default/pods -v — Karthik

Ahmet Alp Balkan Ahmet Alp Balkan · Accepted Answer · 2018-04-09T18:56:16

@Janos pointed out the potential problem, however I think you will need an actual Cloud IAM Service Account as well, because you said:

I want to run an application outside the cluster [...]

If you're authenticating to GKE from outside, I believe you can only use the Google IAM identities. (I might be wrong, if so, please let me know.)

In this case, what you need to do:

Create an IAM service account and download a json key file of it.
set GOOGLE_APPLICATION_CREDENTIALS to that file.
either:
- use RBAC like in your question to give permissions to the email address of the IAM Service Account
- use IAM Roles to give the IAM Service Account on the GKE API (e.g. Container Developer role is usually sufficient)
Use kubectl command against the cluster (make sure you have a .kube/config file with the cluster's IP/CA cert beforehand) with the environment variable above, it should work.

Accessing Kubernetes API in GKE using service accounts

2 Answers