Lucene index updation and performance

2
votes

I am working on a job portal site and have been using Lucene for job search functionality. Users will be posting a number jobs on our site on a daily basis.We need to make sure that new job posted is searchable on the site as soon as possible. In this context, how do I update Lucene index when a new job is posted or when an existing job is edited? Can lucene index updating and search work in parallel?

Also,can I know any tips/best practices with respect to Lucene indexing,optimizing,performance etc?

Appreciate ur help!

Thanks!

3
lucene
My tip would be that using Lucene directly from the application layer is almost always the wrong approach. Unless an application specifically needs low-level index access I'd always recommend access at a higher level, i.e. Solr or Elasticsearch. If your app is successful and you need to scale up, managing your own index files will quickly become very time-consuming. – Richard Marr

3 Answers

4
votes

Yes, Lucene can search from and write to an index at the same time as long as no more than 1 IndexWriter writes to it. If you want the new records visible ASAP, have the IndexWriter call the commit() function often (see IndexWriter's JavaDoc for details).

These Wiki pages might also help:

ImproveIndexingSpeed

ImproveSearchingSpeed

0
votes

I have used Lucene.Net on a web site similar to what you are doing. Yes, you can do live indexes, updating to keep everything up to date? What platform are you using Lucene on, .NET, Java?

0
votes

Make sure you create a new IndexSearcher as any additions after an IndexSearcher has been created are not visible to that instance.

A better approach may be to ReOpen the IndexReader if you want to resuse the same index searcher.