Distribute Elasticsearch's primary shards among newly added data nodes

Question

I am looking for a way to automatically distribute Elasticsearch's primary shards among the newly added data nodes.

I know a way to achieve this is by re-indexing after the new nodes are added. Is there any internal configuration or other simpler way by which it can be done automatically once the cluster configurations are altered?

Example Scenario

Initially, the cluster has 3 data nodes and an index with 6 primary shards and 5 replicas. So Elasticsearch distributes 2 primary shards among each node. I increased the number of data node to 6 after a while. The 3 new nodes automatically share the unassigned replicas but not primary shards. Is there a way to achieve this without re-indexing?

Answer 1

You are correct that Elasticsearch won't automatically relocate the primary shards on newly added data nodes, apart from cluster reroute API which @leandrojmp already mentioned in comment.

few other things you can try but as these are risky and you should only done when you have backup and have good control on your operations and I did them to learn more about while playing and breaking ES and had both backup and full control of operation and is a good way to learn internals of ES and different way to solve the issues:

If you have sufficient replicas(in your case you have 5 replicas), you can reduce the replicas to 2, which would cause elasticsearch to rebalance the shards(although not sure if it would cause primary shards to relocate) and again once its done, increase the replicas again to original count.

forcefully for some time(minimum 60 sec) stop the elasticsearch process on 1 data node which would cause replicas shards to get promoted to primary shards and assigned to new data nodes.

IMHO, cluster reroute API is the safest and best bet here.