elasticsearch: All primary shards become inactive when restart a single node cluster

Question

I started a new elasticsearch cluster and added a new index, then I shut it down. When I try to start it again there is no active shards any more. All the shards became inactive and unassigned. I use this index settings below:

"settings": {
            "number_of_shards": 32,
            "number_of_replicas": 3
        }

Here is health output:

{
  "cluster_name" : "sailcraft",
  "status" : "red",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 0,
  "active_shards" : 0,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 128,
  "number_of_pending_tasks" : 0
}

Here is the same problem which has been marked solved. But I don't think that's the solution.

EDIT:

It's the same question but I think that post does not get the right answer. It said that we need 2 more nodes if we have 2 more replica shards. I read all the related docs and there is not any such restrictions.

EDIT2: Solution: he setting index.recovery.initial_shards provides the following: Basically, when using a local gateway, a "shard" will be recovered once a quorum of its copies are found among the nodes in the cluster. Lets take N shard, with 1 replica (2 copies for each shard). The default (quorum) means that once a singe copy of a shard is found, it will be recovered. If you have 2 replicas (3 copies), it will only recover that shard once 2 copies are found. You can set this setting, in case you lost a large number of nodes, and quorum is a too strict setting. You can actually set it on a "live" index (which is in a "red" state, obviously, because not all shards are recovered).

The link you provided is the reason for the red status in your case. — Andrei Stefan
It's the same question but I think that post dose not get the right answer. It said that we need 2 more nodes if we have 2 more replica shards. I read all the related docs and there is not any such restrictions. — nobody0day

Andrei Stefan Andrei Stefan · Accepted Answer · 2015-06-25T09:06:46

Here's the documentation mentioning this. Indeed, it's not evident, but it's there:

index.recovery.initial_shards

When using local gateway a particular shard is recovered only if there can be allocated quorum shards in the cluster. It can be set to:

quorum (default)

quorum-1 (or half)

full

full-1.

Number values are also supported, e.g. 1.

If you really want to have your node running, set index.recovery.initial_shards: 1 in elasticsearch.yml file.

elasticsearch: All primary shards become inactive when restart a single node cluster

1 Answers