ECK (Elastic cloud on kubernetes) with elastic version 6.8.8

Question

The containers are not coming up, its failing with the below error. Does eck support elastic 6.8.8?

Caused by: java.lang.IllegalArgumentException: unknown setting [discovery.seed_providers] please check that any required plugins are installed, or check the breaking changes documentation for removed settings

configuration

apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: elasticdev
  namespace: ns-4
spec:
  image: elasticsearch/elasticsearch:6.8.8
  http:
    service:
      spec:
        selector:
          common.k8s.elastic.co/type: elasticsearch
          elasticsearch.k8s.elastic.co/cluster-name: elasticdev
          elasticsearch.k8s.elastic.co/role: client       
        type:
          LoadBalancer      
  nodeSets:
  - config:
      node.data: false
      node.master: true
      node.ingest: false
      node.ml: false
    count: 1
    name: master
    podTemplate:
      metadata:
        labels:
          elasticsearch.k8s.elastic.co/role: master      
      spec:
        containers:
        - env:
          - name: ES_JAVA_OPTS
            value: -Xms1024m -Xmx1024m
          limits:
            cpu: 200m
            memory: 1Gi
          name: elasticsearch
          resources:
            requests:
              cpu: 200m
              memory: 1Gi
        nodeSelector:
          eks.amazonaws.com/nodegroup: general
    volumeClaimTemplates:
    - metadata:
        name: elasticsearch-data
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 2Gi
        storageClassName: gp2-storageclass
  - config:
      node.data: false
      node.master: false
      node.ingest: false
      node.ml: false
      search.remote.connect: false
    count: 1
    name: client
    podTemplate:
      metadata:
        labels:
          elasticsearch.k8s.elastic.co/role: client  
      spec:
        containers:
        - env:
          - name: ES_JAVA_OPTS
            value: -Xms512m -Xmx512m
          limits:
            cpu: 200m
            memory: 1Gi
          name: elasticsearch
          resources:
            requests:
              cpu: 200m
              memory: 1Gi
        volumes:       
          - name: elasticsearch-data
            emptyDir: {}
        nodeSelector:
          eks.amazonaws.com/nodegroup: general
  - config:
      node.data: true
      node.master: false
      node.ingest: true
    count: 1
    name: data
    podTemplate:
      metadata:
        labels:
          elasticsearch.k8s.elastic.co/role: data  
      spec:
        containers:
        - env:
          - name: ES_JAVA_OPTS
            value: -Xms1024m -Xmx1024m
          limits:
            cpu: 1000m
            memory: 2Gi
          name: elasticsearch
          resources:
            requests:
              cpu: 1000m
              memory: 2Gi
        nodeSelector:
          eks.amazonaws.com/nodegroup: general
    volumeClaimTemplates:
    - metadata:
        name: elasticsearch-data
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 3Gi
        storageClassName: gp2-storageclass
  updateStrategy:
    changeBudget:
      maxSurge: 1
      maxUnavailable: 1
  version: 6.8.8

Answer 1

Which version of ECK are you using? Is it a fresh new cluster or an upgrade from an exiting cluster?

I tried to reproduce locally with a minimal 6.8.8 cluster, when running ECK 1.0.1, everything is working fine:

apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: elasticsearch-sample
spec:
  version: 6.8.8
  nodeSets:
  - name: default
    count: 3
    config:
      node.store.allow_mmap: false

Answer 2

I noticed that you are setting the Docker image explicitly. That's normally not necessary unless you are using custom Docker images or a private registry. So I am wondering if it is possible that you are using a custom Docker image and that the value of image and version did not align at some point?

To explain a bit more: discovery.seed_providers is a setting which only exists in version 7.0 and above of Elasticsearch. So I am wondering if it is possible that you had at some point a Docker image for 6.8.8 specified but the version attribute was 7.x? The operator would then try to use settings relevant for 7.x and you might see the error you posted.