Lucene Analyzer for a simple direct field search

Question

I tried many lucene analzers and found keyword analyzer to be the best match for my requirement. I am using the same keyword analyzer for both updating the document and searching the same using QueryParser. I want to search for the values with wildcard support. For example : if a field "country" contains the value "india" I can search for the same field as "ind*", "ndi", india etc. I am getting the match for all other searches except the exact match. ie. when i am searching the exact word (country:india), i am not getting any match. If i am changing the same query as "country:india*" or "country:indi?", i am getting the match. Also i have another doubt, if there is a country with the name "not", how can i search for the same. I tried "country:"not"" and "country:\not". But both failed. What is actually happening in both these cases? Please help.

femtoRgon femtoRgon · Accepted Answer · 2013-10-28T17:46:27

I suspect you have some whitespace or other extraneous characters after the country name. You could either trim your input before adding it to Lucene, or implement a custom keyword analyzer, and add a TrimFilter, something like:

public final class CustomKeywordAnalyzer extends Analyzer {
  public CustomKeywordAnalyzer() {
  }

  @Override
  protected TokenStreamComponents createComponents(final String fieldName, final Reader reader) {
    Tokenizer tokenizer = new KeywordTokenizer(reader)
    TokenStream filter = new TrimFilter(Version.LUCENE_43, tokenizer);
    return new TokenStreamComponents(tokenizer, filter);
  }
}

As far as searching for "not", it simply being lowercase should be adequate for it not to be interpreted as a boolean operator (AND, OR, and NOT operators must be uppercase, per the documentation). Those words will get caught by a standard English StopFilter though, such as the one used by StandardAnalyzer. Are you sure you are just using a KeywordAnalyzer when querying?

Barring that, though, the sure way to avoid query parser reserved words would be to just bypass the query parser entirely, and construct the query yourself:

Query query = new TermQuery(new Term("country", userQuery));

Lucene Analyzer for a simple direct field search

1 Answers