Updating lucene index - the most popular way

0
votes

I am updating lucene index onec a day. My strategy in general is:

  1. Find all objects in DB that was modified since last index generation.
  2. Create new tmp-index for these objects. (old index is stil available)
  3. Remove all new indexed Documents (they are in tmp-index) from the old index using IndexWriter.deleteDocuments(Term)
  4. Merge old index and tmp-index using IndexWriter.addIndexes(...)

I have found that in lucene wiki: There is no direct update procedure in Lucene... I have found also that in lucene 4.1.0 doc: A document can be updated with updateDocument...

I have tried IndexWriter.updateDocument(Term, Document) but then performing search with filter I got NPE from one of my methods what not happens when I update index as describe in 1-4. Have anyone had a similar problem? How do you update your index?

2
javaluceneindexing
"got NPE from one of lucene methods" -- can you report this to Lucene JIRA or java-users@ mailing list? Could be you ran into a bug. – mindas
No NPE in lucene - my oversight. – Marcin Sanecki

2 Answers

1
votes

What I do is basically this:

I keep a persistent IndexReader/Readers, this will keep the state that it has since it was created.

I start to delete and create all documents once again. I think I just do a deleteAll() and then recreate them (addDocument()).

I commit, which will activate all those changes.

I drop all IndexReaders that I have, so the next time the system request a Reader, it will create it and store it for subsequent requests.

The updateDocument is basically a delete/create, afaik.

0
votes

You might want to use a SearcherManager to get new IndexSearchers as you update the index with a IndexWriter. I see no need for using a temporary index?