Elasticsearch on Kubernetes - 'Elastic Cloud (ECK)' vs 'Helm charts'

3
votes

For the purpose of log file aggregation, I'm looking to setup a production Elasticsearch instance on an on-premise (vanilla) Kubernetes cluster.

There seems to be two main options for deployment:

  1. Elastic Cloud (ECK) - https://github.com/elastic/cloud-on-k8s
  2. Helm Charts - https://github.com/elastic/helm-charts

I've used the old (soon to be deprecated) helm charts successfully but just discovered ECK.

What are the benefits and disadvantages of both of these options? Any constraints or limitations that could impact long-term use?

2
elasticsearchkuberneteselastic-stackkubernetes-helm

2 Answers

2
votes

The main difference is that the Helm Charts are pretty unopinionated while the Operator is opinionated — it has a lot of best practices built in like a hard requirement on using security. Also the Operator Framework is built on the reconcilliation loop and will continuously check if your cluster is in the desired state or not. Helm Charts are more like a package manager where you run specific commands (install a cluster in version X with Y nodes, now add 2 more nodes, now upgrade to version Z,...).

If ECK is Cloud-on-Kubernetes, you can think of the Helm charts as Stack-on-Kubernetes. They're a way of defining exact specifications running our Docker images in a Kubernetes environment.

Another difference is that the Helm Charts are open source while the Operator is free, but uses the Elastic License (you can't use it to run a paid Elasticsearch service is the main limitation).

0
votes

1. Elastic Cloud (ECK):

ADVANTAGES

  • document oriented (JSON)
  • multilingual - the ICU plugin is used to index and tokenize multilingual content which is an elasticsearch plugin based on the lucene implementation of the unicode text segmentation standard
  • managing and monitoring multiple clusters
  • upgrading to new stack versions with ease
  • scaling cluster capacity up and down
  • changing cluster configuration
  • dynamically scaling local storage (includes Elastic Local Volume, a local storage driver)
  • scheduling backups
  • secure by default - have encryption enabled and are protected with a strong default password right at creation time
  • free features - Canvas, Maps, Uptime
  • hot-warm-cold and custom topologies
  • official GKE support
  • free tier

DISADVANTAGES

  • it is not as good at being a data store as some other options like MongoDB, Hadoop, etc. For smaller use cases, it will perform fine. If you are streaming TB’s of data every day, you will find that it either chokes or loses data
  • it’s learning curve is much steeper
  • when you can’t or won’t create a production-worthy setup because of economics. For test and dev, a single node will work fine. When you move to production, you should have no less than a 3-node/2-replica

More information you can find here: ECK.

2. Elastic Stack Kubernetes Helm Charts:

ADVANTAGES

  • huge community
  • easy to deploy and use in Kubernetes
  • each component in the stack takes care of a different step in the logging pipeline, and together, they all provide a comprehensive and powerful logging solution for Kubernetes
  • rich analysis capabilities

DISADVANTAGES

  • difficult to maintain at scale

More information you can find here: open-source-monitoring-tools-for-kubernetes.