How to create only one instance of `LBHttpSolrServer` for several Solr cores?

Question

I am currently developping a Solr Client with SolrJ using LBHttpSolrServer as I have several Solr nodes.

The problem is that I have an application with more than 200 Solr cores. So, I created an instance of LBHttpSolrServer per Solr core (as seen in the docs):

LBHttpSolrServer solrServer = new LBHttpSolrServer(
    "http://host1:8080/solr/mySolrCore",
    "http://host2:8080/solr/mySolrCore"
);

But, it makes more than 200 instances of LBHttpSolrServer in my application.

As seen in the docs, if a node goes down, SolrJ will create a thread per instance of LBHttpSolrServer, so more than 200 additional threads, and saturate my application.

Is there a way to create only one instance of LBHttpSolrServer which will be shared by all my cores?

Greg S Greg S · Accepted Answer · 2015-02-12T07:00:52

Yes, that's unfortunately how LBHttpSolrServer is implemented:

  /** The provided httpClient should use a multi-threaded connection manager */
  public LBHttpSolrServer(HttpClient httpClient, ResponseParser parser, String...solrServerUrl) {
    clientIsInternal = (httpClient == null);
    this.parser = parser;
    if (httpClient == null) {
      ModifiableSolrParams params = new ModifiableSolrParams();
      params.set(HttpClientUtil.PROP_USE_RETRY, false);
      this.httpClient = HttpClientUtil.createClient(params);
    } else {
      this.httpClient = httpClient;
    }
    for (String s: solrServerUrl) {
      ServerWrapper wrapper = new ServerWrapper(makeServer(s));
      aliveServers.put(wrapper.getKey(), wrapper);
    }
    updateAliveList();
  }

What you can do to work around that is to use a different request method that is provided by LBHttpSolrServer:

public Rsp request(Req req)

instead of the default one that is overwritten from SolrServer

public NamedList<Object> request(final SolrRequest request)

request (Req req) method provided by LBHttpSolrServer allows specifying a list of Solr URLs via Req parameter

"@param req contains both the request as well as the list of servers to query"

in this case LBHttpSolrServer will only keep instantiating HttpSolrServers until one works:

for (String serverStr : req.getServers()) {
  serverStr = normalize(serverStr);
  // if the server is currently a zombie, just skip to the next one
  ServerWrapper wrapper = zombieServers.get(serverStr);
  if (wrapper != null) {
    // System.out.println("ZOMBIE SERVER QUERIED: " + serverStr);
    if (skipped.size() < req.getNumDeadServersToTry())
      skipped.add(wrapper);
    continue;
  }
  rsp.server = serverStr;
  HttpSolrServer server = makeServer(serverStr);

  ex = doRequest(server, req, rsp, isUpdate, false, null);
  if (ex == null) {
    return rsp; // SUCCESS
  }

How to create only one instance of `LBHttpSolrServer` for several Solr cores?

1 Answers