Write Lucene index concurrently

Question

An index is shared by multiple IndexWriters. Obviously, LockObtainFailedException will be thrown if an opening index is about to open by other IndexWriter. My solution is to create IndexWriter with long timeout:

IndexWriterConfig conf= new IndexWriterConfig(Version.LUCENE_30, new SimpleAnalyzer (Version.LUCENE_30));
conf.setWriteLockTimeout(30*1000);//Wait 30 seconds timeout
try{
        IndexWriter writer = new IndexWriter(dir, conf);
}catch(LockObtainFailedException e){
        System.out.println("give up trying...");
}

Any better solution for this case?

EDIT: inspired by Thilo. I found JavaDoc of IndexWriter:

IndexWriter instances are completely thread safe, meaning multiple threads can call any of its methods, concurrently. If your application requires external synchronization, you should not synchronize on the IndexWriter instance as this may cause deadlock; use your own (non-Lucene) objects instead.

Better solution is to only use one server process for writing and have everyone go through that. Your approach is okay if you have low concurrency (i.e. almost never have to wait). Maybe take a look at Solr. — Thilo
@Thilo Do you mean that multi Threads shares a single IndexWriter? — 卢声远 Shengyuan Lu
Yes, like a database connection pool (except that it is a pool of one). If this is multiple threads in the same JVM (and no other process writing to the index), you could do what you do now, and just synchronize the threads. — Thilo
You don't even have to synchronize anything, because IndexWriter is thread-safe... — Matthijs Bierman

Matthijs Bierman Matthijs Bierman · Accepted Answer · 2012-02-02T10:13:52

A single Directory cannot be shared by multiple IndexWriters. Just share one IndexWriter amongst your threads.

Write Lucene index concurrently

2 Answers