Docker in Docker permissions error

18
votes

I have a docker in docker setup for CI. Essentially, the machine has a jenkins CI server on it that uses the same machines docker socket to create nodes for CI.

This was working great until I recently updated docker. I've identified the issue, but I can't seem to figure out the right magic to get it working.

host $ docker exec -it myjenkins bash
jenkins@container $ docker ps
Got permission denied while trying to connect to the Docker daemon 
socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.26/containers/json: dial unix /var/run/docker.sock: connect: permission denied

host $ docker exec -it -u root -it myjenkins bash
root@container $ docker ps 
... docker ps from host container yay! ...

So here's what I surmise. I have access to the host docker socket from within the container, but I can't seem to give permission to the jenkins user.

I've added the docker group, and also added the jenkins user to the docker group. But I still get the same error. I've restarted a whack of times so, I'm kind of at a loss for what to do next.

Is there a way to force permissions for a user on a particular socket?

4
jenkinsdocker
Just out of curiosity, did you try the same thing with a privileged container (as root inside)? – Nehal J Wani
If you look at my question where I present console. I showed that I tried as the jenkins user, and then again as the root user of the container. The root user has access, the jenkins user does not. – Brandon Bertelsen
Just out of curiosity, did you try the same thing with a privileged container (as jenkins inside) – Nehal J Wani
And the same thing with ` --userns=host` ? – Nehal J Wani
It is a privliedged container. I'm not sure how to user --userns=host how would I specify that using docker-compose. – Brandon Bertelsen

4 Answers

27
votes

You need to map the gid of the docker group on your host to the gid of a group that jenkins belongs to inside your container. Here's a sample from my Dockerfile of how I've built a jenkins slave image:

ARG DOCKER_GID=993

RUN groupadd -g ${DOCKER_GID} docker \
  && curl -sSL https://get.docker.com/ | sh \
  && apt-get -q autoremove \
  && apt-get -q clean -y \
  && rm -rf /var/lib/apt/lists/* /var/cache/apt/*.bin 

RUN useradd -m -d /home/jenkins -s /bin/sh jenkins \
  && usermod -aG docker jenkins

The 993 happens to be the gid of docker on the host in this example, you'd adjust that to match your environment.

Solution from the OP: If rebuilding isn't a possibility you can set the docker group accordingly in using root and add the user. If you tried this before you may have to delete the group on the slave (groupdel docker): 

docker exec -it -u root myjenkins bash
container $ groupadd -g 993 docker
container $ usermod -aG docker jenkins
2
votes

As mentioned in other answers, you must ensure that user "jenkins" inside the container has permission to issue Docker commands to the Docker engine on the host via the /var/run/docker.sock mount.

An easy way to do this is: 

$ docker run --rm -d --group-add $(stat -c '%g' /var/run/docker.sock) -v /var/run/docker.sock:/var/run/docker.sock -P myjenkins

But beware:

If your Jenkins job has the need to execute Docker build/run commands (e.g., to build a container), this is not sufficient. The reason is that the containerized Jenkins will ask the Docker in the host to deploy a Docker agent container and that Docker agent container will face similar "permission errors" when connecting to the Docker daemon on the host.

There is a blog post describing the problem and solution for this at https://blog.nestybox.com/2019/09/29/jenkins.html.

1
votes

Personally, i've just had to do this :

sudo gpasswd -a $USER docker

And it worked out

0
votes

Execute the command with your root user by including sudo in the start.