kubernetes securitycontext runAsNonRoot Not working

3
votes

I am testing with securityContext but I cant start a pod when I set runAsNonRoot to true. I use vagrant to deploy a master and two minions and ssh to the host machine as the user abdelghani :

id $USER
uid=1001(abdelghani) gid=1001(abdelghani) groups=1001(abdelghani),27(sudo)

Cluster information:

Kubernetes version: 4.4.0-185-generic Cloud being used: (put bare-metal if not on a public cloud) Installation method: manual Host OS: ubuntu16.04.6 CNI and version: CRI and version:

apiVersion: v1
kind: Pod
metadata:
  name: buggypod
spec:
  containers:
  - name: container
    image: nginx
    securityContext:        
      runAsNonRoot: true

I do : kubectl apply -f pod.yml it says pod mybugypod created but when I check with : kubectl get pods the pod’s status is CreateContainerConfigError

what is it I am doing wrong?

2
securitykubernetes
Can you please run "kubectl describe pod <Pod Name>" and share that output with us? It's unclear why it's failing right now. Don't forget to wrap the output in some formating so we can read it. - embik
As Pointed out by @Rohit it is because nginx image needs to start as root. I tried with busybox, it works at the condition that user exists within the image (for instance user 65534) - Abdelghani

2 Answers

1
votes

Nginx service will expect a read and write permission to its configuration path (/etc/nginx) by default non root user would have that access to the path that is the reason it is failing. You just set runAsNonRoot but you can't expect or guarantee that container will start the service as user 1001. Please try setting runAsUser explicitly to 1001 like below, this should resolve your issue.

apiVersion: v1
kind: Pod
metadata:
  name: buggypod
spec:
  containers:
    - name: container
      image: nginx
      securityContext:        
        runAsUser: 1001
0
votes

I try to run the pod based on your requirement. And the reason it failed is the Nginx require to modify some configuration in /etc/ owned by root and when you runAsNonRoot it fails as it cannot edit the Nginx default config.

This is the error you actually get when you run it.

10-listen-on-ipv6-by-default.sh: error: can not modify /etc/nginx/conf.d/default.conf (read-only file system?)
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2020/08/13 17:28:55 [warn] 1#1: the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /etc/nginx/nginx.conf:2
nginx: [warn] the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /etc/nginx/nginx.conf:2
2020/08/13 17:28:55 [emerg] 1#1: mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied)
nginx: [emerg] mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied)

The spec I ran.

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: buggypod
  name: buggypod
spec:
  securityContext:
    runAsNonRoot: true
    runAsUser: 1000
  containers:
  - image: nginx
    name: buggypod
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

My suggestion is you create a custom Nginx image with a Dockerfile that also creates user and provides permissions to the folders /var/cache/nginx, /etc/nginx/conf.d, /var/log/nginx for the newly created user. Such that you achieve running the container as Non-Root.