lucene search in iOS inconsistent

2
votes

I'm trying to improve a search algorithm of an iOS app, which offers some magazines with a lot of articles. To do that I used the S4LuceneLibrary of Micheal Papp (git repo), which is a n iOS equivalent to the full-featured text search engine library of Apache Lucene. The problem is, that the search now is very incosistent...that means the search after specific words takes sometimes very long and on the other hand sometimes not.

That's a list of words, I was searching for, and the time the search took:

  • Berlin (34 hits) -> 3,5 seconds
  • Tag (29 hits) -> 8,3 seconds
  • Haus (3 hits) -> 3,6 seconds
  • Straße (28 hits) -> 8 seconds
  • Raumfahrt (5 hits) -> 6,2 seconds
  • Astronomie (9 hits) -> 1 second

So the results are kind of different, but I thought it should take the same time for every search phrase. Do you know where is the difference?

Thanks in advance :)

2
iossearchlucene
These response times are very high especially for simple term queries... I can't believe there is not something really wrong either in the library code or in the client code. Although I am familiar with Lucene, I am not familiar at all with iOS. Is there any profiling tool available on iOS that would help you find the bottleneck? - jpountz
Yes, they are kind of high, but if you compare the times, they are differentiating with a maximal factor of 8, and that's confusing me. So the word "Astronomie" just takes 1 sec, and a word like "Tag" takes more than 8 secs. Why is that? - tobi_b

2 Answers

0
votes

Wow, those benchmarks! how big is your index/collection?

I'm assuming here, you're searching your index programmatically? (and not via some complex implementation/UI, which might have glitches of its own)

My first port of call would be to ensure you properly indexed your collection in the first place. Especially: Check the constructors of the fields you put in each document. Some forms of the constructor take a boolean for 'store' and 'index': be sure you've passed it: store.yes and index.yes in this context, otherwise they'll be there and retrievable, but not via lucene's inverted index - the crux of the entire tool.

However, given that your collection probably isn't changing in size, if this is the case, it's strange that you're getting dramatically different search times.

Could you show us how you populate/Query your index?

0
votes

Okay, for testing I have 6 magazines and each of them has its own index. So the index sizes are:

434kb, 41kb, 139kb, 434kb, 57kb and 57kb

My code for indexing is:

LCSimpleAnalyzer *analyzer = [[LCSimpleAnalyzeralloc] init];
    LCIndexWriter *writer = [[LCIndexWriteralloc] initWithDirectory:[selfcreateFileDirectoryWithEmagPath:emagModel.emagPath] analyzer: analyzer create: YES];

...
// creating searchResult
...

// adding relevant information to lucene document
LCDocument *doc = [[LCDocumentalloc] init];

LCField *fieldContent = [[LCFieldalloc] initWithName: @"content_to_search"string:bodyText store:LCStore_YES index:LCIndex_Tokenized];   
LCField *fieldResult = [[LCFieldalloc] initWithName:@"data" data: [NSKeyedArchiverarchivedDataWithRootObject:result] store:LCStore_YES];

[doc addField:fieldContent];
[doc addField:fieldResult];  

[writer addDocument:doc];

...
// releasing stuff and close writer

And that's the code for searching:

// search
LCIndexSearcher *searcher = [[LCIndexSearcheralloc] initWithDirectory: [selfgetFileDirectoryOfEmagPath:emagPath]];
LCTerm *term = [[LCTermalloc] initWithField: @"content_to_search" text: self.searchText];
LCTermQuery *termQuery = [[LCTermQueryalloc] initWithTerm:term];

LCHits *hits = [searcher search:termQuery];

thx