2
votes

I am trying to add --admission-control=ServiceAccount to my kube-apiserver call to be able to host a https connection from the kubernetes-ui and the apiserver. I am getting this on the controller manager.

Mar 25 18:39:51 master kube-controller-manager[1388]: I0325 18:39:51.425556 1388 event.go:211] Event(api.ObjectReference{Kind:"ReplicaSet", Namespace:"default", Name:"nginx4-3088538572", UID:"aefae1a6-f2b8-11e5-8269-0401bd450a01", APIVersion:"extensions", ResourceVersion:"252", FieldPath:""}): type: 'Warning' reason: 'FailedCreate' Error creating: pods "nginx4-3088538572-" is forbidden: no API token found for service account default/default, retry after the token is automatically created and added to the service account

RIght now my default serviceaccount looks like this

cesco@desktop: ~/code/go/src/bitbucket.org/cescoferraro/cluster/terraform on master [+!?]
$ kubectl get serviceaccount default -o wide
NAME      SECRETS   AGE
default   0         2m
cesco@desktop: ~/code/go/src/bitbucket.org/cescoferraro/cluster/terraform on master [+!?]
$ kubectl get serviceaccount default -o json
{
    "kind": "ServiceAccount",
    "apiVersion": "v1",
    "metadata": {
        "name": "default",
        "namespace": "default",
        "selfLink": "/api/v1/namespaces/default/serviceaccounts/default",
        "uid": "eaa3c6e1-f2cd-11e5-973f-0401bd52ec01",
        "resourceVersion": "30",
        "creationTimestamp": "2016-03-25T21:09:52Z"
    }
}

I am using a token to authenticate to kubernetes and the full cluster is works on https.

CONTROLLER-MANAGER

ExecStart=/opt/bin/kube-controller-manager \
                              --address=0.0.0.0 \
                              --root-ca-file=/home/core/ssl/ca.pem \
                              --service-account-private-key-file=/home/core/ssl/kube-key.pem  \
                              --master=https://${COREOS_PRIVATE_IPV4}:6443 \
                              --logtostderr=true \
                              --kubeconfig=/home/core/.kube/config  \
                              --cluster-cidr=10.132.0.0/16 \
                              --register-retry-count 100

APISERVER

ExecStart=/opt/bin/kube-apiserver \
                          --admission-control=NamespaceLifecycle,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota \
                          --logtostderr=true  \
                          --insecure-bind-address=${MASTER_PRIVATE} \
                          --insecure-port=8080  \
                          --bind-address=0.0.0.0  \
                          --secure-port=6443  \
                          --runtime-config=api/v1 \
                          --allow-privileged=true \
                          --service-cluster-ip-range=10.100.0.0/16 \
                          --advertise-address=${MASTER_PUBLIC} \
                          --token-auth-file=/data/kubernetes/token.csv \
                          --etcd-cafile=/home/core/ssl/ca.pem   \
                          --etcd-certfile=/home/core/ssl/etcd1.pem  \
                          --etcd-keyfile=/home/core/ssl/etcd1-key.pem \
                          --etcd-servers=https://${MASTER_PRIVATE}:2379,https://${DATABASE_PRIVATE}:2379 \
                          --cert-dir=/home/core/ssl \
                          --client-ca-file=/home/core/ssl/ca.pem \
                          --tls-cert-file=/home/core/ssl/kubelet.pem \
                          --tls-private-key-file=/home/core/ssl/kubelet-key.pem \
                          --kubelet-certificate-authority=/home/core/ssl/ca.pem \
                          --kubelet-client-certificate=/home/core/ssl/kubelet.pem \
                          --kubelet-client-key=/home/core/ssl/kubelet-key.pem \
                          --kubelet-https=true

.kube/config

ExecStart=/opt/bin/kubectl config set-cluster CLUSTER  \
                                        --server=https://${MASTER_PRIVATE}:6443 \
                                        --certificate-authority=/home/core/ssl/ca.pem
ExecStart=/opt/bin/kubectl config set-credentials admin  \
                                        --token=elezxaMiqXVcXXU7lRYZ4akrlAtxY5Za \
                                        --certificate-authority=/home/core/ssl/ca.pem \
                                        --client-key=/home/core/ssl/kubelet-key.pem \
                                        --client-certificate=/home/core/ssl/kubelet.pem
ExecStart=/opt/bin/kubectl config set-context default-system \
                                        --cluster=CLUSTER \
                                        --user=admin
ExecStart=/opt/bin/kubectl config use-context default-system

UPDATE 1

Per @Jordan Liggitt answer, I added --service-account-key-file=/home/core/ssl/kubelet-key.pem to the apiserver call but now I am getting

Mar 26 11:19:30 master kube-apiserver[1874]: F0326 11:19:30.556591    1874 server.go:410] Invalid Authentication Config: asn1: structure error: tags don't match (16 vs {class:0 tag:2 length:1 isCompound:false}) {optional:false explicit:false application:false defaultValue:<nil> tag:<nil> stringType:0 set:false omitEmpty:false} tbsCertificate @2
2
Shouldn't that match the file name passed to the controller manager (kube-key.pem)?Jordan Liggitt
I have tested the way you said, and the error persists. Now I am using the same certificate in all the flags at all kubernetes daemons. And the error still persists.CESCO
Got it now. Now I am using cluster/saltbase/salt/generate-cert/make-ca-cert.sh to make the certs and these issue are gone. I haven't quite got the dashboard to work yet, but at least the default secret is being created and mounted on every pod.CESCO

2 Answers

2
votes

Make sure you start the controller manager with a service account key (used to sign generated service account tokens) and start the API server with the corresponding public key (used to verify the tokens during auth)

3
votes

With 1.6 version, you can auto mount the token if you mention it while creating the service account like this:

apiVersion: v1 kind: ServiceAccount metadata: name: sysdig automountServiceAccountToken: true