traefik setup on gke not working

2
votes

I’m trying to get traefik running in GKE, following the user guide (https://docs.traefik.io/user-guide/kubernetes/).

Instead of seeing the dashboard, I get a 404. I guess there’s a problem with the RBAC setup somewhere but I can’t figure it out.

Any help would be greatly appreciated.

The ingress controller log shows a constant flow of (one each second):

E0714 12:19:56.665790 1 reflector.go:205] github.com/containous/traefik/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Service: services is forbidden: User "system:serviceaccount:kube-system:traefik-ingress-controller" cannot list services at the cluster scope: Unknown user "system:serviceaccount:kube-system:traefik-ingress-controller"

and the traefik pod itself constantly spews:

E0714 12:17:45.108356 1 reflector.go:205] github.com/containous/traefik/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.Ingress: ingresses.extensions is forbidden: User "system:serviceaccount:default:default" cannot list ingresses.extensions in the namespace "kube-system": Unknown user "system:serviceaccount:default:default"

E0714 12:17:45.708160 1 reflector.go:205] github.com/containous/traefik/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Service: services is forbidden: User "system:serviceaccount:default:default" cannot list services in the namespace "default": Unknown user "system:serviceaccount:default:default"

E0714 12:17:45.714057 1 reflector.go:205] github.com/containous/traefik/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Endpoints: endpoints is forbidden: User "system:serviceaccount:default:default" cannot list endpoints in the namespace "kube-system": Unknown user "system:serviceaccount:default:default"

E0714 12:17:45.714829 1 reflector.go:205] github.com/containous/traefik/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.Ingress: ingresses.extensions is forbidden: User "system:serviceaccount:default:default" cannot list ingresses.extensions in the namespace "default": Unknown user "system:serviceaccount:default:default"

E0714 12:17:45.715653 1 reflector.go:205] github.com/containous/traefik/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Endpoints: endpoints is forbidden: User "system:serviceaccount:default:default" cannot list endpoints in the namespace "default": Unknown user "system:serviceaccount:default:default"

E0714 12:17:45.716659 1 reflector.go:205] github.com/containous/traefik/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Service: services is forbidden: User "system:serviceaccount:default:default" cannot list services in the namespace "kube-system": Unknown user "system:serviceaccount:default:default"

I created the clusterrole using:

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: traefik-ingress-controller
rules:
  - apiGroups: [""]
    resources: ["servies", "endpoints", "secrets"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["extensions"]
    resources: ["ingresses"]
    verbs: ["get", "list", "watch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: traefik-ingress-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
  name: traefik-ingress-controller
  namespace: kube-system

and then deployed traefik as deployment:

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
  labels:
    k8s-app: traefik-ingress-lb
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: traefik-ingress-lb
  template:
    metadata:
      labels:
        k8s-app: traefik-ingress-lb
        name: traefik-ingress-lb
    spec:
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 60
      containers:
      - image: traefik
        name: traefik-ingress-lb
        ports:
        - name: http
          containerPort: 80
        - name: admin
          containerPort: 8080
        args:
        - --api
        - --kubernetes
        - --logLevel=INFO
---
kind: Service
apiVersion: v1
metadata:
  name: traefik-ingress-service
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
    - protocol: TCP
      port: 80
      name: web
    - protocol: TCP
      port: 8080
      name: admin
  type: LoadBalancer

when using helm to install traefik I used the following values file:

dashboard:
  enabled: true
  domain: traefik.example.com
kubernetes:
  namespaces:
    - default
    - kube-system

and finally, for the UI I used the following yaml:

---
apiVersion: v1
kind: Service
metadata:
  name: traefik-web-ui
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
  - name: web
    port: 80
    targetPort: 8080
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: traefik-web-ui
  namespace: kube-system
spec:
  rules:
  - host: traefik.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: traefik-web-ui
          servicePort: web

thanks for looking!

(edit: corrected typo in title)

2
kubernetestraefikkubernetes-ingressgoogle-kubernetes-engine

2 Answers

2
votes

Since the namespace "kube-system" is handled by the Master node, you will not be able to deploy anything on that specific namespace. The Master node within GKE is a managed service and is not accessible to users at this time.

If you would like to have this functionality, then the only suggestion I can provide at this time is to create your own custom cluster from scratch. This will allow you to have access to the Master Node and you would have the option to customize your cluster to your liking.

Edit: I was able to find instructions from github on how to use Traefik as a GKE loadbalancer. I would suggest testing this first before running it in your production cluster.

1
votes

I think your problem is that you're setting up a ClusterRoleBinding with name "traefik-ingress-controller" and namespace "kube-system" but Traefik is running in namespace default with serviceaccount default.

Try changing your ClusterRoleBinding to:

kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: traefik-ingress-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
  name: default
 namespace: default

Or deploy your system with serviceaccount "traefik-ingress-controller" and in namespace "kube-system"