Lucene.net limitations on the index files with large data

Question

We are planning to use to lucene.net for search in order to have a google like search. We will have a huge amount of data which will be loaded from d/b which needs to be indexed. Is there any limitations on size of the indexing documents and what is the maximum size of a single index document be? How to distribute the size on an average size of each document?

Would be much easier if you defined the "huge amount of data" you have, what are you talking about ? GB TB PB scale? — Ronan Thibaudau
The size of the files would be in GB as the records in the database would be in lakhs. — mehalok

Ronan Thibaudau Ronan Thibaudau · Accepted Answer · 2014-09-15T18:40:26

There are no limitations with the data sizes you're hinting at, i'm running a lucene.net app that works with 100s of gb of indexed data without spliting the index (outside of how lucene naturally splits it without you asking).

Just add your data to your index and forget about any limitations, you're way bellow any potential issues. (but do read all their performance guidelines, the lucene ones as lucene.net is a direct port, all those tips apply).

Lucene.net limitations on the index files with large data

1 Answers