Escaping special characters in lucene and query using wildcard

Question

I have an issue when I try to query using wildcard in a term that has a special character in it. As an example if I index "Test::Here",I search using this using wildcard ? for "TE?T\:\:Here" (NOTE: I escaped ':'). I do not get any results. I use standard analyser and queryparser for indexing and searching.

Anyone encountered similar issue?

Answer 1

StandardAnalyzer uses StandardTokenizer, so Test::Here is seen as two tokens: Test and Here. Wildcard queries are not run through an analyzer, so you end up matching colons against the terms that do not contain them. You need to use different tokenizer, for example WhitespaceTokenizer.

Answer 2

You can't search what you haven't indexed. Below is a code to see what you index.

var analyzer = new AnyAnalyzer();
TokenStream tokensTream = analyzer.TokenStream("", new StringReader("Test::Here"));
Lucene.Net.Analysis.Token token = tokensTream.Next();
while (token != null)
{
    Console.Write("[" + token.TermText() + "] ");
    token = tokensTream.Next();
}

Answer 3

Artur is right, but there is another issue to consider which is that wildcard terms are not analyzed at all in Lucene, so you will have to make sure that the case of your query term matches the case of the indexed term (after analysis).