Error on pagination of huge result set

0
votes

I perform a text search on Solr which generates a huge result set like 2,277,292.

I show only 20 documents/page. As I click on the last page, here at back end a query to solr is generated as

start=2277280
rows=20

An exception is raised:

HTTP Status 500 - allocLargeObjectOrArray: [Lorg/apache/lucene/search/ScoreDoc;, size 9109184 java.lang.OutOfMemoryError: allocLargeObjectOrArray: [Lorg/apache/lucene/search/ScoreDoc;, size 9109184 at org.apache.lucene.search.TopDocsCollector.topDocs(TopDocsCollector.java:132) at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:996) at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:884) at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:341) at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:182) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:619)

1
solr
can you please format the exception to multiple lines – Mauricio Scheffer
does this happen with all queries (i.e. also with a trivial *:* query) or just with a specific query? If so, what's this specific query? – Mauricio Scheffer
no it happens with the queries that end up with very huge result set – Ahsan Iqbal
*:* would generate a huge result set (i.e. all documents in the index), so does this also happen with *:* ? – Mauricio Scheffer

1 Answers

0
votes

Not sure if this is a bug in Solr, but you could just try to increase the memory:

set 

JAVA_OPTS="-Xmx1024m -Xms256m"

in catalina.sh/.bat