How to configure Harbor on k8s cluster?

Question

I'm trying to deploy Harbor(private docker registry) on k8s cluster(provisioned by Rancher) using helm(values.yaml). below is my valuse.yaml.

Network
I have nginx front of k8s ingress nginx controller.
nginx(http 301 redirect to https, ex. http://harbor.mydomain.com -> https://harbor.ssgadm.com) -> ingress-nginx -> harbor service
Harbor with nginx
https://github.com/goharbor/harbor/blob/master/docs/1.10/install-config/troubleshoot-installation.md#using-nginx-or-load-balancing
Harbor team says if Harbor is running behind an nginx proxy, remove header 'X-Forwarded-Proto', so I added ingress annotation in values.yaml

Here are my problems.
1. succeeded login docker, but failed to push docker image with message : denied: requested access to the resource is denied

cannot see minio registry endpoint at harbor web page
couldn't find any endpoints!

So, could you help me solve problems?

---
  harborAdminPassword: "admin"
  secretKey: "add-your-secret0"
  logLevel: "info"
  expose: 
    type: "ingress"
    ingress: 
      hosts: 
        core: "harbor.mydomain.com"
        notary: "harbor.notary.mydomain.com"
      annotations:
        ingress.kubernetes.io/proxy-body-size: "0"
        ingress.kubernetes.io/ssl-redirect: "true"
        nginx.ingress.kubernetes.io/proxy-body-size: "0"
        nginx.ingress.kubernetes.io/ssl-redirect: "true"
        nginx.org/server-snippets: |
          location / {
            proxy_hide_header 'X-Forwarded-Proto'
          }

          location /v2/ {
            proxy_hide_header 'X-Forwarded-Proto'
          }

          location /service/ {
            proxy_hide_header 'X-Forwarded-Proto'
          }
    loadBalancer: 
      IP: ""
    tls: 
      enabled: false
      secretName: ""
  externalURL: "https://harbor.mydomain.com"
  persistence: 
    imageChartStorage: 
      type: "s3"
      disableredirect: true
      azure: 
        accountname: ""
        accountkey: ""
        container: ""
      gcs: 
        bucket: ""
        encodedkey: ""
      s3: 
        bucket: "docker"
        region: "us-east-1"
        accesskey: "minio"
        secretkey: "minio"
        regionendpoint: "https://minio.mydomain.com"
      oss: 
        bucket: ""
        region: ""
        accesskeyid: ""
        accesskeysecret: ""
      swift: 
        authurl: "https://storage.myprovider.com/v3/auth"
        username: ""
        password: ""
        container: ""
    enabled: true
    persistentVolumeClaim: 
      registry: 
        storageClass: "nfs-provisioner"
        size: "1Gi"
        existingClaim: ""
      chartmuseum: 
        storageClass: "nfs-provisioner"
        size: "1Gi"
        existingClaim: ""
      jobservice: 
        storageClass: "nfs-provisioner"
        size: "1Gi"
        existingClaim: ""
      database: 
        storageClass: "nfs-provisioner"
        size: "1Gi"
        existingClaim: ""
      redis: 
        storageClass: "nfs-provisioner"
        size: "1Gi"
        existingClaim: ""
  jobservice: 
    jobLogger: "database"
  database: 
    type: "internal"
    external: 
      host: ""
      username: ""
      password: ""
      coreDatabase: "registry"
      clairDatabase: "clair"
      notaryServerDatabase: "notary_server"
      notarySignerDatabase: "notary_signer"
      sslmode: "disable"
      port: "5432"
  redis: 
    type: "internal"
    external: 
      host: ""
      port: "6379"
      password: ""
  clair: 
    enabled: true
  notary: 
    enabled: false
  chartmuseum: 
    enabled: true
  cert: 
    enabled: true

Malgorzata Malgorzata · Accepted Answer · 2020-02-28T09:58:17

Make sure that you have all prerequisites are fulfilled.

Make sure that you have provided certificates to Harbor and Docker.

After generating the ca.crt, yourdomain.com.crt, and yourdomain.com.key files, you must provide them to Harbor and to Docker, and reconfigure Harbor to use them. Finally restart Docker Engine.

To enable pushing images to docker registry please execute following commands:

You need to include the namespace for Docker Hub to associate it with your account. The namespace is the same as your Docker Hub account name. You need to rename the image to YOUR_DOCKERHUB_NAME/docker-whale.

Tag image before pushing:

$ docker tag your_image YOUR_DOCKERHUB_NAME/your_image

and then you should be able to push it.

$ docker push YOUR_DOCKERHUB_NAME/your_image

You have also duplicated nginx.ingress.kubernetes.io/proxy-body-size annotation in configuration file.

For NGINX, an 413 error will be returned to the client when the size in a request exceeds the maximum allowed size of the client request body. This size can be configured by the parameter client_max_body_size.

To configure this setting globally for all Ingress rules, the proxy-body-size value may be set in the NGINX ConfigMap. To use custom values in an Ingress rule define these annotation:

nginx.ingress.kubernetes.io/proxy-body-size: 8m

try to add 0m.

Also environment sanity check: should fail without SSL, enable external ss in database:

sslmode=require

If more problem occur check logs of installed harbor:

$ kubectl logs -n harbor your-harbor-pod

How to configure Harbor on k8s cluster?

1 Answers