load balancing elasticsearch nodes for document / index writes and updates?

Question

I am architecting a medium sized elasticsearch cluster (~20 nodes) with the three distinct node types Elasticsearch allows (master, data, and client)

I am working on setting up load balancers to receive new documents and document updates for the Elasticsearch cluster.

I have found Elasticsearch documentation on routing a document to a shard. But have not found if this routing takes place on a specific node type (i.e. only master nodes perform document routing; or if master and data nodes perform document routing)?

Another way of asking this is: will any node type in the Elasticsearch cluster perform a shard routing lookup when a document write is received and route the write/update request to the correct shard -- or does only the master node perform shard routing lookups for document writes/updates?

Knowing which node types perform document to shard routing is important to the load balancer setup.

Thank you :)

Answer 1

To answer your question i need to start from cluster state. Cluster state has all information about the cluster mainly what all index exists , how many shard each index has and on which nodes these shards are maintained.

This cluster state , though maintained by master node , all the nodes has a copy of it. Which means the master is never a single point of failure , any node can route an index call to the right shard. Now only the master node can make any change to the cluster state. This means that if a new index is made , or if a new machines comes or goes from cluster , the master node make the change to the cluster state and broadcasts them to all the nodes.

Even if the master goes down , some other node would be elected as master.

Now coming back to your question routing can happen on any node and there is no point in load balancer , at-least while indexing. The index request no matter to which node it hits , would be routed to a single machine to which document should go. By creating several shards , you are already applying a load balancing logic there.