Lucene near real time search

Question

I am using Lucene 6.6.0 and I would like to use the near real-time search feature of Lucene. However, I could not manage to implement it. The way I try to get the feature is as follows:

I initialize an IndexReader instance:

this.reader = DirectoryReader.open(this.directory);

Let's assume some changes have been made in the index via an IndexWriter instance. Then, if I understand correctly, I need a second instance of IndexReader to commit updates:

this.newReader = DirectoryReader.openIfChanged(this.reader);
if (this.newReader != null) {
    // Update the IndexSearcher with the new IndexReader instance
    this.searcher = new IndexSearcher(this.newReader);
    this.reader.close();
}

The issue here is that the code does not compile because of the following error: The method openIfChanged(DirectoryReader) in the type DirectoryReader is not applicable for the arguments (IndexReader).

How should I update the IndexReader then ?

Secondly, if I update the index again, I will need another IndexReader instance, won't I ? Would the most optimal way to update the index freely during the execution of the program be by switching between 2 IndexReader instances after each update ?

Thank you.

dom dom · Accepted Answer · 2017-07-24T09:04:03

Try to use a SearcherManager instead of a IndexReader: http://lucene.apache.org/core/6_6_0/core/org/apache/lucene/search/SearcherManager.html

Based on the SearcherManager your able to execute following methods:

// get a IndexSearcher for searching
IndexSearcher searcher = searcherManager.aquire();

// release IndexSearcher after search
searcherManager.release(searcher);

// refresh and add new index records to next search. usually after a commit 
searcherManager.maybeRefresh();

I tried to implement this as well and basically i did this:

create an IndexWriter and leave it open
create a SearcherManager with the IndexWriter as param.
use SearcherManager to search
use IndexWriter for indexing operations.
commit after indexing

Additionally you can use a separate thread to commit periodically and not on every write because the commit operation may be pretty "expensive".

Example here: http://www.lucenetutorial.com/lucene-nrt-hello-world.html

Lucene near real time search

1 Answers