0
votes

I want to create a customized docker image and be able to use kubernetes to pull my customized docker image from private docker registry. Here are my setups:

Environment: docker registry ip:10.179.143.115 kubernetes master ip: 10.179.143.113

  1. generate a certificate:
curl -O https://raw.githubusercontent.com/driskell/log-courier/1.x/src/lc-tlscert/lc-tlscert.go

go build lc-tlscert.go

./lc-tlscert

mkdir certs

mv selfsigned.* certs/
  1. Create docker registry:

docker run -d --restart=always --name registry -v `pwd`/certs:/certs -e REGISTRY_HTTP_ADDR=0.0.0.0:443 -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/selfsigned.crt -e REGISTRY_HTTP_TLS_KEY=/certs/selfsigned.key -p 443:443 registry:2

  1. Create my customized docker vm(just tag the vm with another name for testing purpose)
docker pull tomcat

docker tag tomcat 10.179.143.115/test-tomcat

docker push 10.179.143.115/test-tomcat
  1. On Kubernetes master:
copy selfsigned.*(crt and key file)  to /usr/local/share/ca-certificates/

sudo update-ca-certificates

sudo service docker restart

root@kubernetes-master:~# docker images

REPOSITORY                                               TAG                 IMAGE ID            CREATED             SIZE
gcr.io/google_containers/kube-apiserver-amd64            v1.9.3              360d55f91cbf        3 weeks ago         210 MB
gcr.io/google_containers/kube-controller-manager-amd64   v1.9.3              83dbda6ee810        3 weeks ago         138 MB
gcr.io/google_containers/kube-proxy-amd64                v1.9.3              35fdc6da5fd8        3 weeks ago         109 MB
gcr.io/google_containers/kube-scheduler-amd64            v1.9.3              d3534b539b76        3 weeks ago         62.7 MB
quay.io/coreos/flannel                                   v0.10.0-amd64       f0fad859c909        5 weeks ago         44.6 MB
gcr.io/google_containers/etcd-amd64                      3.1.11              59d36f27cceb        2 months ago        194 MB
gcr.io/google_containers/k8s-dns-sidecar-amd64           1.14.7              db76ee297b85        4 months ago        42 MB
gcr.io/google_containers/k8s-dns-kube-dns-amd64          1.14.7              5d049a8c4eec        4 months ago        50.3 MB
gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64     1.14.7              5feec37454f4        4 months ago        41 MB
gcr.io/google_containers/pause-amd64                     3.0                 99e59f495ffa        22 months ago       747 kB

root@kubernetes-master:~# docker pull 10.179.143.115/test-tomcat
Using default tag: latest
latest: Pulling from test-tomcat
f0f063e89695: Pull complete
d9b7671d4a80: Pull complete
6eb55822688c: Pull complete
a85cc2721f25: Pull complete
ee9e2e7b610a: Pull complete
562dd1fb5637: Pull complete
e8e2e3cceeee: Pull complete
86cbf3cde839: Pull complete
3678522c43a2: Pull complete
50ea7ae5efa3: Pull complete
e81b257a8ae8: Pull complete
5b298dc937bc: Pull complete
Digest: sha256:332fa1b89534f0b0e45c636a26edb8520b15bcdfc05ef5450efae3e71d1b1361
Status: Downloaded newer image for 10.179.143.115/test-tomcat:latest

5.However, when I want to create a kubernete pod:

 test.yaml: 
    apiVersion: v1 
    kind: Pod 
    metadata:   
    name: test 
    spec:  
    containers:
      - name: test
        image: 10.179.143.115/test-tomcat
kubectl create -f test.yaml

root@kubernetes-master:~# kubectl describe pods test

Name:         test
Namespace:    default
Node:         kubernetes-node/10.179.143.114
Start Time:   Fri, 02 Mar 2018 15:02:20 -0500
Labels:       <none>
Annotations:  <none>
Status:       Pending
IP:
Containers:
  test:
    Container ID:
    Image:          10.179.143.115/test-tomcat
    Image ID:
    Port:           <none>
    State:          Waiting
      Reason:       ErrImagePull
    Ready:          False
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-lvz9r (ro)
Conditions:
  Type           Status
  Initialized    True
  Ready          False
  PodScheduled   True
Volumes:
  default-token-lvz9r:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-lvz9r
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason                 Age               From                      Message
  ----     ------                 ----              ----                      -------
  Normal   Scheduled              32s               default-scheduler         Successfully assigned test to kubernetes-node
  Normal   SuccessfulMountVolume  31s               kubelet, kubernetes-node  MountVolume.SetUp succeeded for volume "default-token-lvz9r"
  Normal   Pulling                9s (x2 over 21s)  kubelet, kubernetes-node  pulling image "10.179.143.115/test-tomcat"
  Warning  Failed                 9s (x2 over 21s)  kubelet, kubernetes-node  Failed to pull image "10.179.143.115/test-tomcat": rpc error: code = Unknown desc = Error response from daemon: Get https://10.179.143.115/v1/_ping: x509: certificate signed by unknown authority
  Warning  Failed                 9s (x2 over 21s)  kubelet, kubernetes-node  Error: ErrImagePull
  Normal   SandboxChanged         9s (x2 over 20s)  kubelet, kubernetes-node  Pod sandbox changed, it will be killed and re-created.

6.Error message is :

Failed to pull image "10.179.143.115/test-tomcat": rpc error: code = Unknown desc = Error response from daemon: Get https://10.179.143.115/v1/_ping: x509: certificate signed by unknown authority

Plesase bear with my bad formatting and thanks in advance!

2

2 Answers

3
votes

Thanks for all the help guys! Here is a follow up on how I make it works.

When I copy all the certs to kubenetes master, I am able to pull and push docker images from my private registry. But it was not working when I want to create kubernetes pods. And I realized that I need also to copy all the certs to my kubernetes slave and that is the place where kubernetes actually pull the images from private docker registry. After I copy the certs to "/usr/local/share/ca-certificates/" and run "sudo update-ca-certificates; sudo service docker restart" , I am able to create pods now!

-1
votes

In my understanding, a Secret resource has to be created to use the private docker registry. Refer Kubernetes documentation!