Can Elasticsearch version 5.5.3 run on Kubernetes with StatefulSet?

3
votes

I've been trying to get Elasticsearch running on K8s using the newly-promoted-to-stable helm chart, which works fine, BTW, for elasticsearch v 6.4.2. However, we're tied to a grails app that requires elasticsearch v 5.5.3, for which we don't have the ability to upgrade. I've downgraded the elasticsearch image version in the chart to 5.5.3 (and also tried v 5.6.12) but it fails to start.

I looked into the elasticsearch operator, but it's currently set up to work with AWS S3 storage types, out of the box, and GCP with a little work (although no snapshot ability). Before I dive into this, I'd like to know if it will work with 5.5.3, to begin with.

Does anyone know if I can get elasticsearch v 5.5.3 running in a k8s cluster? I would say using a k8s StatefulSet at a minimum.

Thanks!

Update

I suppose I should have given the errors that the existing helm chart is having when downgrading elasticsearch image to 5.5.3.

master-0 pod fails to start with:

Error injecting constructor, ElasticsearchException[java.io.IOException: failed to read [id:15, legacy:false, file:/usr/share/elasticsearch/data/nodes/0/_state/global-15.st]]; nested: IOException[failed to read [id:15, legacy:false, file:/usr/share/elasticsearch/data/nodes/0/_state/global-15.st]]; nested: ElasticsearchException[Unknown license version found, please upgrade all nodes to the latest elasticsearch-license plugin]; at org.elasticsearch.gateway.GatewayMetaState.<init>(Unknown Source) while locating org.elasticsearch.gateway.GatewayMetaState for parameter 4 at org.elasticsearch.gateway.GatewayService.<init>(Unknown Source) while locating org.elasticsearch.gateway.GatewayService Caused by: ElasticsearchException[java.io.IOException: failed to read [id:15, legacy:false, file:/usr/share/elasticsearch/data/nodes/0/_state/global-15.st]]; nested: IOException[failed to read [id:15, legacy:false, file:/usr/share/elasticsearch/data/nodes/0/_state/global-15.st]]; nested: ElasticsearchException[Unknown license version found, please upgrade all nodes to the latest elasticsearch-license plugin];

The client pods fail with:

[2018-10-22T17:52:51,835][WARN ][o.e.d.z.UnicastZenPing ] [elasticsearch-client-6bf954c595-7zlpc] failed to resolve host [elasticsearch-discovery] java.net.UnknownHostException: elasticsearch-discovery

Clearly, it's expecting a later elasticsearch version.

2
elasticsearchkuberneteskubernetes-helm
back port the helm chart to version 5.5.3 – Ijaz Ahmad Khan
I'm not sure what you mean @ljaz-Khan? There is no helm chart for that version of elasticsearch. And, as I said, when I change the elasticsearch image version to 5.5.3, if fails to start -- the {elasticsearch-discovery} fails to start and {Unknown license version found, please upgrade all nodes to the latest elasticsearch-license plugin}. I was wondering if someone knew of other changes to the helm chart that might work. Long shot, I know. Thanks. – Jeff Cunningham
FYI, I was looking into the history of helm chart to see if there was any point where it supported v5.5.3. It looks like there moved from v5.4 to v6.6.1 here – Puneet Behl

2 Answers

1
votes

The short answer here is that ElasticSearch 5.5.3 should work with Kubernetes. Note the configs for 5.5.3 are slightly different, I believe they changed after 5.6 where for example the made x-pack enabled by default, and yes use a StatefulSet or start from the existing stable Helm chart.

0
votes

We ended up going with a variation of this Medium post to get ES 5.5.3 working in "multi-node" mode. This is not HA in the way the latest ES chart version provides but will get us on our way.

We used a regular storage class, instead of the SSD and we had to disable ES boot checking, for now, because we couldn't get the ulimit command to work on ES init.

We have multiple ES nodes and are good until we can upgrade the app to use ES latest version.