I am using SOLR cloud with Tomcat. When one of my leader nodes is down(e.g. Shard 1); its replica is not elected as leader, but is still up. I am using a stand-alone zookeeper(well, 3 of them). When I restart the failed node, the node is not visible, and still no leader for Shard 1 is elected. The replica is still up and leader-less. This remains so even if I restart zookeeper and all nodes.
I am using SOLR 4.7.0 The setup I have is: 1 tomcat instance per shard. I have 2 shards and two replica shards. The replicas are on separate servers. I use an external zookeeper instance to manage all four nodes/cores. I have also tested with 3 external zk instances, but had the same result.
Can anyone tell me where to look if this is a configuration issue or sounds familiar?