Elasticsearch 6.3.0 on Kubernetes

1
votes

I am trying to deploy production grade Elasticsearch 6.3.0 on Kubernetes.

Came across few articles, but still not sure what is the best approach to go with.

  1. https://github.com/pires/kubernetes-elasticsearch-cluster

It doesn't use stateful set.

  1. https://anchormen.nl/blog/big-data-services/elastic-search-deployment-kubernetes/

This is pretty old.

Using elastic search for App search.

Images from Elasticsearch are

docker pull docker.elastic.co/elasticsearch/elasticsearch:6.3.0
docker pull docker.elastic.co/elasticsearch/elasticsearch-oss:6.3.0

I would like to go with -oss image and it is the core Apache one.

Is there any good documentation on setting up production grade 6.3.0 version on Kubernetes.

1
elasticsearchkubernetes
Do you really need a StatefulSet? We are running an ElasticSearch cluster based on Deployments like the one by pires just fine. I think his documentation on Github and his manifests are a solid base to start from and you can just change the Deployment to a StatefulSet if you believe it is the better choice for you. – matthias krull
The proper production grade setup and configuration is dependent on your use case anyway :) – matthias krull
If not stateful, what will happen to the data when the pod gets restarted or when deployed to other pod for some other reason. – user1578872
what will happen to the data when the pod gets restarted or when deployed to other pod for some other reason conflates two concepts that are not related to one another: pod restarts, and persistent data. A Pod restart is zero drama, so long as the volumes: are correctly declared because that's kubernetes' job, StatefulSet or Deployment will behave the same. The difference between the two is whether a volume is tied to a Pod's identity, such as db-0 always needs to start up with db-0's data, in contrast to "fine, startup with any data so long as it persists over time" – mdaniel
I also will say that in regard to your "production grade ES cluster" question, ensure and I mean ensure that your volume provider has enough IOPS to support an ES cluster, because those things will thrash a disk for sure. In AWS, you'll want io1 with as many provisioned IOPS as you can afford, because it is bad news to have an ES cluster driving the load average high on a Node because of the blocking I/O – mdaniel

1 Answers

1
votes

One of the most promising new developments for running Elasticearch on Kubernetes is the Elasticsearch Operator.

Kubernetes Operators allow for more sophistication when it comes to dealing with the requirements of complex tools (and Elasticsearch is definitely one). Especially when considering the need to avoid losing Elasticsearch data, an operator is the way to go.