Can't deploy kubernetes dashboard (HTTPS) publicly

0
votes

im new to kubernetes and after researching a long time it seems im running out of resources so I'd be more than glad if somebody could help me.

What I currently have

  • One Node that i am connecting to via SSH, bare-metal kubernetes setup (Notice that im not running k8s in the cloud)
  • kubectl version yields 1.20 on Client and Server Side
  • NGINX Ingress Controller
  • Cluster setup with kubeadm
  • Following Pods, Services and one ingress resource (dashboard):

Pods:

NAMESPACE NAME READY STATUS RESTARTS AGE
ingress-nginx ingress-nginx-admission-create-4ml4p 0/1 Completed 0 23m
ingress-nginx ingress-nginx-admission-patch-jj9c6 0/1 Completed 1 23m
ingress-nginx ingress-nginx-controller-67897c9494-kxwgv 1/1 Running 0 23m
kube-system coredns-74ff55c5b-2xbvq 1/1 Running 0 23m
kube-system coredns-74ff55c5b-sc667 1/1 Running 0 23m
kube-system etcd-k8s.mydomain.de 1/1 Running 0 23m
kube-system kube-apiserver-k8s.mydomain.de 1/1 Running 0 23m
kube-system kube-controller-manager-k8s.mydomain.de 1/1 Running 0 23m
kube-system kube-flannel-ds-fhzzp 1/1 Running 0 23m
kube-system kube-proxy-lq7tt 1/1 Running 0 23m
kube-system kube-scheduler-k8s.mydomain.de 1/1 Running 0 23m
kubernetes-dashboard dashboard-metrics-scraper-7b59f7d4df-v6qsn 1/1 Running 0 23m
kubernetes-dashboard kubernetes-dashboard-74d688b6bc-5x4nd 1/1 Running 0 23m

Services:

NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default kubernetes ClusterIP 10.96.0.1 none 443/TCP 33d
ingress-nginx ingress-nginx-controller NodePort 10.98.67.34 none 80:32346/TCP, 443:30710/TCP 33d
ingress-nginx ingress-nginx-controller-admission ClusterIP 10.110.196.58 none 443/TCP 33d
kube-system kube-dns ClusterIP 10.96.0.10 none 53/UDP,53/TCP, 9153/TCP 33d
kubernetes-dashboard dashboard-metrics-scraper ClusterIP 10.109.128.22 none 8000/TCP 33d
kubernetes-dashboard kubernetes-dashboard ClusterIP 10.98.158.159 none 443/TCP 33d

Ingress Resource (Dashboard):

NAMESPACE NAME CLASS HOSTS ADDRESS PORTS AGE
kubernetes-dashboard dashboard-ingress dashboard.k8s.mydomain.de 10.10.1.164 80 52m

When I try to reach the kubernetes dashboard using https://dashboard.k8s.mydomain.de nothing happens. When I try to reach it using https://dashboard.k8s.mydomain.de:30710, which is the HTTPS TCP Port of the nginx ingress controller, then it works. Its a non secure connection and the browser warns me, but it works. But obviously this is not the desired behavior.

What i want to have

I want to deploy the kubernetes dashboard in a way so that https://dashboard.k8s.mydomain.de gives access to the kubernetes dashboard over HTTPS. I don't know why this is currently not the case.

These are the configuration .yaml files i used for the deployments and stuff:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: dashboard-ingress
  namespace: kubernetes-dashboard
  annotations:
    nginx.ingress.kubernetes.io/backend-protocol: HTTPS
spec:
  rules:
    - host: dashboard.k8s.mydomain.de
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: kubernetes-dashboard
                port:
                  number: 443

Logs of the nginx ingress controller tell me that it registers the ingress resource, so i am confused:

NGINX Ingress controller
  Release:       v0.44.0
  Build:         f802554ccfadf828f7eb6d3f9a9333686706d613
  Repository:    https://github.com/kubernetes/ingress-nginx
  nginx version: nginx/1.19.6
I0323 09:42:57.665847       6 flags.go:208] "Watching for Ingress" class="nginx"
W0323 09:42:57.665953       6 flags.go:213] Ingresses with an empty class will also be processed by this Ingress controller

-------------------------------------------------------------------------------

W0323 09:42:57.667132       6 client_config.go:614] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0323 09:42:57.667958       6 main.go:241] "Creating API client" host="https://10.96.0.1:443"
I0323 09:42:57.682847       6 main.go:285] "Running in Kubernetes cluster" major="1" minor="20" git="v1.20.3" state="clean" commit="01849e73f3c86211f05533c2e807736e776fcf29" platform="linux/amd64"
I0323 09:42:57.997597       6 main.go:105] "SSL fake certificate created" file="/etc/ingress-controller/ssl/default-fake-certificate.pem"
I0323 09:42:58.003204       6 main.go:115] "Enabling new Ingress features available since Kubernetes v1.18"
W0323 09:42:58.008105       6 main.go:127] No IngressClass resource with name nginx found. Only annotation will be used.
I0323 09:42:58.033445       6 ssl.go:532] "loading tls certificate" path="/usr/local/certificates/cert" key="/usr/local/certificates/key"
I0323 09:42:58.070414       6 nginx.go:254] "Starting NGINX Ingress controller"
I0323 09:42:58.077773       6 event.go:282] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"ingress-nginx", Name:"ingress-nginx-controller", UID:"a76c1e40-f5aa-4353-aeea-35bccbbb57a7", APIVersion:"v1", ResourceVersion:"3497961", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap ingress-nginx/ingress-nginx-controller
I0323 09:42:59.183747       6 event.go:282] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"kubernetes-dashboard", Name:"dashboard-ingress", UID:"6f02c931-bfad-44a1-a219-4e2b2970365e", APIVersion:"networking.k8s.io/v1beta1", ResourceVersion:"3497191", FieldPath:""}): type: 'Normal' reason: 'Sync' Scheduled for sync
I0323 09:42:59.271684       6 nginx.go:296] "Starting NGINX process"
I0323 09:42:59.271826       6 leaderelection.go:243] attempting to acquire leader lease ingress-nginx/ingress-controller-leader-nginx...
I0323 09:42:59.272447       6 nginx.go:316] "Starting validation webhook" address=":8443" certPath="/usr/local/certificates/cert" keyPath="/usr/local/certificates/key"
I0323 09:42:59.272851       6 controller.go:146] "Configuration changes detected, backend reload required"
I0323 09:42:59.288414       6 leaderelection.go:253] successfully acquired lease ingress-nginx/ingress-controller-leader-nginx
I0323 09:42:59.288501       6 status.go:84] "New leader elected" identity="ingress-nginx-controller-67897c9494-kxwgv"
I0323 09:42:59.302119       6 status.go:201] "POD is not ready" pod="ingress-nginx/ingress-nginx-controller-67897c9494-kxwgv" node="k8s.mydomain.de"
I0323 09:42:59.307191       6 status.go:281] "updating Ingress status" namespace="kubernetes-dashboard" ingress="dashboard-ingress" currentValue=[{IP:10.10.1.164 Hostname: Ports:[]}] newValue=[]
I0323 09:42:59.315053       6 event.go:282] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"kubernetes-dashboard", Name:"dashboard-ingress", UID:"6f02c931-bfad-44a1-a219-4e2b2970365e", APIVersion:"networking.k8s.io/v1beta1", ResourceVersion:"3498054", FieldPath:""}): type: 'Normal' reason: 'Sync' Scheduled for sync
I0323 09:42:59.381846       6 controller.go:163] "Backend successfully reloaded"
I0323 09:42:59.382014       6 controller.go:174] "Initial sync, sleeping for 1 second"
I0323 09:42:59.382266       6 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"ingress-nginx", Name:"ingress-nginx-controller-67897c9494-kxwgv", UID:"2ba6f14e-36a8-401b-a1d8-00921cbe9604", APIVersion:"v1", ResourceVersion:"3497997", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configuration
I0323 09:43:59.323910       6 status.go:281] "updating Ingress status" namespace="kubernetes-dashboard" ingress="dashboard-ingress" currentValue=[] newValue=[{IP:10.10.1.164 Hostname: Ports:[]}]
I0323 09:43:59.333091       6 event.go:282] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"kubernetes-dashboard", Name:"dashboard-ingress", UID:"6f02c931-bfad-44a1-a219-4e2b2970365e", APIVersion:"networking.k8s.io/v1beta1", ResourceVersion:"3498144", FieldPath:""}): type: 'Normal' reason: 'Sync' Scheduled for sync

When i set up the cluster with microk8s instead of kubeadm it works. Please specify what I have to do in order to make this work.

Thanks in Advance

2
kuberneteskubectlnginx-ingresskubernetes-dashboard

2 Answers

2
votes

Please run nginx ingress controller in hostNetwork Mode https://kubernetes.github.io/ingress-nginx/deploy/baremetal/#via-the-host-network

template:
  spec:
    hostNetwork: true

And run it as DaemonSet.

And set dnsPolicy to ClusterFirstWithHostNet

Also, please read the security warnings in https://kubernetes.github.io/ingress-nginx/deploy/baremetal/#via-the-host-network

If you are using https://kubernetes.github.io/ingress-nginx/deploy/#using-helm then add below to values file

controller:
  dnsPolicy: ClusterFirstWithHostNet
  hostNetwork: true
  kind: DaemonSet
0
votes

i am not sure what you are actually getting at the place of ingress resource : dashboard.k8s.mydomain.de

apart of that in your service : ingress-nginx-controller is exposed as NodePort it should be as the LoadBalancer so that you can get one IP.

You can add that IP in to DNS as A record and map the domain. Inside ingress you can add the domain same you did.

Now for HTTPS you might have to create and store the certificate in secret so that Nginx ingress can use the certificate and server the traffic over HTTPS.

if you are using the cert-manager for auto manging the TLS/SSL certificate ayour ingress will be something like this :

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: dev
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/proxy-body-size: 50m
    nginx.ingress.kubernetes.io/proxy-read-timeout: "2000"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "2000"
    nginx.ingress.kubernetes.io/rewrite-target: /
  name: dev-ingress
spec:
  rules:
  - host: dev.example.io
    http:
      paths:
      - backend:
          serviceName: service
          servicePort: 80
  tls:
  - hosts:
    - dev.example.io
    secretName: dev