Jenkins + Kubenetes: How to use kubectl in kubernetes-plugin

Question

I am configuring Jenkins on Kubernetes system. It works fine to build. But in order to deploy, we need to call kubectl or helm. Currently, I am using

lachlanevenson/k8s-kubectl:v1.8.8
lachlanevenson/k8s-helm:latest

It is fail and throw exception: "Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:jenkins:default" cannot list pods in the namespace "jenkins""

The jenkins script is simple:

def label = "worker-${UUID.randomUUID().toString()}"

podTemplate(label: label,containers: [
  containerTemplate(name: 'kubectl', image: 'lachlanevenson/k8s-kubectl:v1.8.8', command: 'cat', ttyEnabled: true)
],
volumes: [
  hostPathVolume(mountPath: '/var/run/docker.sock', hostPath: '/var/run/docker.sock')
]){
  node(label) {
    stage('Run kubectl') {
        container('kubectl') {
            sh "kubectl get pods"
        }    
    }
  }
}

Could you please let me know what is wrong?

Thanks,

Looks like an RBAC issue. Have you checked if you have adequate role defined for jenkins user in your cluster — fatcook

Cosmic Ossifrage Cosmic Ossifrage · Accepted Answer · 2018-09-06T12:27:16

The Kubernetes (k8s) master, as of Kubernetes v1.8, by default implements role-based access control (RBAC) security controls on accesses to its API. The RBAC controls limit access to the k8s API by your workloads to only those resources and methods which you have explicitly permitted.

You should create a role which permits access to the pod resource's list verb (and any other resources you require¹), create a service account object, and finally create a role binding which assigns the role to the service account.

Finally, provide the service account to your Jenkins deployment by supplying its name in the serviceAccountName property of the Pod template. Ensure automountServiceAccountToken is true to have k8s install an API key into your Pod. Attempts to access the k8s API using the native k8s API wrappers and libraries should find this key and automatically authenticate your requests.

¹_{If you are planning to make deployments from Jenkins, you will certainly require more than the ability to list Pods, as you will be required to mutate objects in the system. However, if you use Helm, it is Helm's Tiller pod which influences the downstream k8s objects for your deployments, so the set of permissions you require for the Helm Tiller and for Jenkins to communicate with the Tiller will vary.}

Jenkins + Kubenetes: How to use kubectl in kubernetes-plugin

1 Answers