Setting up ELK stack on kubernetes using minikube

0
votes

After installing this is what my pods look like

Running pods
NAME                                          READY   STATUS                  RESTARTS   AGE
elk-elasticsearch-client-5ffc974f8-987zv      1/1     Running                 0          21m
elk-elasticsearch-curator-1582107120-4f2wm    0/1     Completed               0          19m
elk-elasticsearch-data-0                      0/1     Pending                 0          21m
elk-elasticsearch-exporter-84ff9b656d-t8vw2   1/1     Running                 0          21m
elk-elasticsearch-master-0                    1/1     Running                 0          21m
elk-elasticsearch-master-1                    1/1     Running                 0          20m
elk-filebeat-4sxn9                            0/2     Init:CrashLoopBackOff   9          21m
elk-kibana-77b97d7c69-d4jzz                   1/1     Running                 0          21m
elk-logstash-0                                0/2     Pending                 0          21m

So filebeat refuses to start. Getting the logs from this node I get 

Exiting: Couldn't connect to any of the configured Elasticsearch hosts. Errors: [Error connection to Elasticsearch http://elk-elasticsearch-client.elk.svc:9200: Get http://elk-elasticsearch-client.elk.svc:9200: lookup elk-elasticsearch-client.elk.svc on 10.96.0.10:53: no such host]

Also when trying to access the kibana node (the only node i can call using http) I get that it is not ready.

get pv:

pvc-9b9b13d8-48d2-4a79-a10c-8d1278554c75   4Gi        RWO            Delete           Bound    default/data-elk-elasticsearch-master-0   standard                113m
pvc-d8b361d7-8e04-4300-a0f8-c79f7cea7e44   4Gi        RWO            Delete           Bound    default/data-elk-elasticsearch-master-1   standard                112m

I'm running minikube with the none vm-driver which it tells me, does not respect the memory or cpu-flag. But I don't get it complaining about resources

kubectl version 1.17
docker version i 19.03.5, build 633a0ea838
minikube version 1.6.2

The elk stack was installed using helm. I have the following versions:

  • elasticsearch-1.32.2.tgz
  • elasticsearch-curator-2.1.3.tgz
  • elasticsearch-exporter-2.2.0.tgz
  • filebeat-4.0.0.tgz
  • kibana-3.2.6.tgz
  • logstash-2.4.0.tgz

Running on ubuntu 18.04

1
dockerkubernetesminikube
No, I am not sure that I've given enough information. I just try my best. – Tobbes
Layout has been improved :) – Tobbes
can you please provide the reason why elk-filebeat-4sxn9 is failing to start?. Do this oc describe pod elk-filebeat-4sxn9 and let me know the last couple of lines. – Rajat Singh
The Running pods clearly shows that pods are still pending. This seems like lack of resources. Can you get output from commands: kubectl describe pod for elk-elasticsearch-data-0 and elk-logstash-0 pods? This might have information why are they still pending. filebeat most likely is waiting for elasticsearch to be ready. – Piotr Malec
Why are you using tarballs instead of APT packages or existing Elastic containers? – OneCricketeer

1 Answers

1
votes

Tearing everything down and then installing the required components from other helm-charts solved the issues. It may be that the charts I was using were not intended to run locally on minikube.