Solr Index pull fails

Question

I have master - slave configuration of solr.On master index will be updated 24X7.its a continuous process.my slave pulls the index from the master after every 30 minutes.

The Problem is this index pull is failing because of the following exception

SnapPull failed : org.apache.solr.common.SolrException: Index fetch failed : at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:329) [:1.4.1 955763M - mark - 2010-06-17 18:06:42] at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:264) [:1.4.1 955763M - mark - 2010-06-17 18:06:42] at org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:159) [:1.4.1 955763M - mark - 2010-06-17 18:06:42] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) [:1.6.0_26] at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) [:1.6.0_26] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) [:1.6.0_26] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) [:1.6.0_26] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) [:1.6.0_26] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) [:1.6.0_26] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [:1.6.0_26] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [:1.6.0_26] at java.lang.Thread.run(Thread.java:662) [:1.6.0_26] Caused by: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: SimpleFSLock@/data/solr/data/index/lucene-47510fa3e31f1e71e7408a128e1ebac1-write.lock at org.apache.lucene.store.Lock.obtain(Lock.java:85) [:2.9.3 951790 - 2010-06-06 01:30:55] at org.apache.lucene.index.IndexWriter.init(IndexWriter.java:1565) [:2.9.3 951790 - 2010-06-06 01:30:55] at org.apache.lucene.index.IndexWriter.(IndexWriter.java:1421) [:2.9.3 951790 - 2010-06-06 01:30:55] at org.apache.solr.update.SolrIndexWriter.(SolrIndexWriter.java:191) [:1.4.1 955763M - mark - 2010-06-17 18:06:42] at org.apache.solr.update.UpdateHandler.createMainIndexWriter(UpdateHandler.java:98) [:1.4.1 955763M - mark - 2010-06-17 18:06:42] at org.apache.solr.update.DirectUpdateHandler2.openWriter(DirectUpdateHandler2.java:173) [:1.4.1 955763M - mark - 2010-06-17 18:06:42] at org.apache.solr.update.DirectUpdateHandler2.forceOpenWriter(DirectUpdateHandler2.java:376) [:1.4.1 955763M - mark - 2010-06-17 18:06:42] at org.apache.solr.handler.SnapPuller.doCommit(SnapPuller.java:471) [:1.4.1 955763M - mark - 2010-06-17 18:06:42] at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:319) [:1.4.1 955763M - mark - 2010-06-17 18:06:42] ... 11 more

Is this exception because of continuous index updation on master or any other reasons for this.how can i get rid of this.

Thanks in advance.

Regards Ram

Answer 1

I think the explanation by jpountz is correct. I was able to resolve the issue by shutting down the solr web server (Jetty in my case) and deleting the entire solr index dir, then restarting. Deleting the lock file alone did not correct the problem.

Answer 2

There is a problem with a lock (/data/solr/data/index/lucene-47510fa3e31f1e71e7408a128e1ebac1-write.lock) on your slave. This can happen, for example, when Solr crashes while an IndexWriter is open : after the following restart, the lock file has not been cleared so Solr thinks another IndexWriter is updating the index and refuses to perform any change.

To get rif of this error, you can :

• shut Solr down,
• ensure that there is no active lock in your directory (remove the lock files if necessary),
• re-start Solr.

Answer 3

I have also experienced the same issue. I resolved by,

1.Stop Solr and depended applications

2.rm -rf /{path to your solr data dir}/index.*

3.rm /{path to your solr data dir}/replication.properties

4.Start Solr (Wait for 10-20 minutes to replicate based on your indexed data)

Check the status of the Solr cloud and also trace the log file of solr.