Is there a way to index multiple Solr cores simultaneously?

1
votes

I am developing an indexing application using Solr. Our current system has two live cores and indexes only one core at a time. It has recently become apparent that the current indexing system will not work long term. One of the live cores needs to be split into two new cores. They will have some overlapping information, but different schemas. Both will need to be updated quickly whenever a new project is ingested into the database.

Is there a way to simultaneously update multiple solr cores using SolrJ?

All cores are in the same solr instance.

We are not using SolrCloud.

The core that needs to be split currently contains approx. 2500000 documents.

Any help is appreciated.

2
solrsolrj
what does it mean simultaneously in this case? For sure you could fire up requests at the same time to update cores – Mysterion
I want to index two cores at the same time from the same database, or at least one immediately after the other. The current system starts up, connects to solr, indexes, disconnects, and terminates the application. It ends up running the entire application from start to finish multiple times to index each core. I don't see why I can't index them all in one application run. – ceh

2 Answers

0
votes

Since you are indexing many documents on a single core I would assume the indexing process takes quite some time and using all system resources ( if configured correctly ). In that case - parallel indexing on the same instance will not help as your multiple threads will be sharing the same resources.

But what you could do is index another core on another instance and then do replication of each core separately.

0
votes

When you build a Solr client using SolrJ it's specific to the core and not to your complete Solr instance. Having said that you could have multiple process updating any number of cores in your application.