I am using Lucene in a web application. Lucene is used to create index when an article is added. Since only one IndexWriter instance may exist for an index directory, I use a singleton IndexWriter for the application.
when should I close IndexWriter to commit all documents to the index directory?
If you are much worried about the data loss, commit as frequently as you can (can do even after each document). You can still keep old reader(s) hanging around and reopen them later if the search freshness isn't crucial.
if the web server(tomcat) goes down, how should I recover the uncommitted documents? Has Lucene offered any ways to recover from a crash?
Lucene is atomic - as soon as you exit writer.commit() or writer.close(), the changes are persisted to the disk and can be read afterwards. I think this should be enough.
Don't forget: Lucene is only a search library. It is your responsibility to ensure the application doesn't crash. Even if it does, it is your responsibility to implement a recovery strategy. Lucene already does a pretty decent job with regards to atomicity.
