Is it possible to run the elk stack on kubernetes using minikube

1
votes

I am trying to get the elk stack running locally on a linux ubuntu 18.04 laptop using minikube. But I don' get past step one, the elastiearch master nodes do not come on line. Appearantly because the physical volumes are not being created, or cannot be used. My short question is whether minikube is supported as a means of running kibana/elastisearch/logstach or are they solely intented to be running in the clould. I've been following this tutorial:

https://logz.io/blog/deploying-the-elk-stack-on-kubernetes-with-helm/

Running pods
NAME                                          READY   STATUS                  RESTARTS   AGE
elk-elasticsearch-client-5ffc974f8-987zv      1/1     Running                 0          21m
elk-elasticsearch-curator-1582107120-4f2wm    0/1     Completed               0          19m
elk-elasticsearch-data-0                      0/1     Pending                 0          21m
elk-elasticsearch-exporter-84ff9b656d-t8vw2   1/1     Running                 0          21m
elk-elasticsearch-master-0                    1/1     Running                 0          21m
elk-elasticsearch-master-1                    1/1     Running                 0          20m
elk-filebeat-4sxn9                            0/2     Init:CrashLoopBackOff   9          21m
elk-kibana-77b97d7c69-d4jzz                   1/1     Running                 0          21m
elk-logstash-0                                0/2     Pending                 0          21m

So filebeat won't start for me I tried doing kubectl logs elk-filebeat-4sxn9 load-es-template

From which I get:

Exiting: Couldn't connect to any of the configured Elasticsearch hosts. Errors: [Error connection to Elasticsearch http://elk-elasticsearch-client.elk.svc:9200: Get http://elk-elasticsearch-client.elk.svc:9200: lookup elk-elasticsearch-client.elk.svc on 10.96.0.10:53: no such host]

1
minikubeelk

1 Answers

2
votes

Well, yes, you can use minikube but you need to install it with correct options.

Keep in mind this is not recommended.

In Elastic docs about Production Deployment - Hardware we can read:

  • MEMORY

    A machine with 64 GB of RAM is the ideal sweet spot, but 32 GB and 16 GB machines are also common. Less than 8 GB tends to be counterproductive

    Minikube by deafault only allocates 2GB of RAM, you can check that by $ minikube config get memory. You can increase this value by using --memory flag or making that persistent with $ minikube config set memory 4096

  • CPU

    This is not really an issue because:

    Most Elasticsearch deployments tend to be rather light on CPU requirements. As such, the exact processor setup matters less than the other resources. You should choose a modern processor with multiple cores. Common clusters utilize two- to eight-core machines.

  • DISKS

    They are important and usually they are a main bottleneck in the cluster.

    If you can afford SSDs, they are by far superior to any spinning media. SSD-backed nodes see boosts in both query and indexing performance. If you can afford it, SSDs are the way to go.

    ...

    If you use spinning media, try to obtain the fastest disks possible (high-performance server disks, 15k RPM drives).

    Using RAID 0 is an effective way to increase disk speed, for both spinning disks and SSD. There is no need to use mirroring or parity variants of RAID, since high availability is built into Elasticsearch via replicas.

Those are the basic stuff related to ELK. You can read ELK Architecture and requirements and Hardware requirement ELK for more details regarding data consumption.

You would need to share the logs from the pods and your minikube configuration so we could start debugging the issue.