ASCIIFoldingFilter usage example on Lucene.NET

Question

I am trying to implement my search on Lucene.NET and my needs are:

search and find the result directly
if there is no result, search again with Accent Insensitive

I did it on SQL Server but I want to move it to Lucene.NET. I made a research and first I found ISOLatinFilter and then ASCIIFoldingFilter in Lucene. But I couldn't find a simple example how to use it (Even in Lucene in Action book)

Can you please give me a small sample code to achieve accent insensitive search? Do I need to change anything else on Indexing? As I need accent sensitive also, I cannot create an Accent insensitive index only.

Thanks

This may give you some ideas: stackoverflow.com/questions/3354948/… — nikhil500
I checked and tried it but when I use ASCIIFoldingFilter, it indexes my values with only accent insensitive versions. (i.e: göz is indexed as goz) But I need both versions. (i.e: When I search for göz it shouldn't return goz if there is a direct göz record) Is there a way to index both Accent sensitive and insensitive versions? — Omer Celik
You need to index the data twice (in 2 different fields) such that one field has accents while the other doesn't. While searching, first look in the field with accents and then in the field without accents. Alternately, you can search in both fields at once but give higher weight to the field with accents. — nikhil500

syb syb · Accepted Answer · 2013-02-11T12:03:50

Use this class as your Analyzer on index and search, Works for me.

    public class CustomAnalyzer : StandardAnalyzer
    {
        Lucene.Net.Util.Version matchVersion;

        public CustomAnalyzer(Lucene.Net.Util.Version p_matchVersion)
            : base(p_matchVersion)
        {
            matchVersion = p_matchVersion;
        }

        public override TokenStream TokenStream(string fieldName, System.IO.TextReader reader)
        {
            TokenStream result = new StandardTokenizer(matchVersion, reader);
            result = new StandardFilter(result);
            result = new ASCIIFoldingFilter(result);
            return result;
        }

    }

ASCIIFoldingFilter usage example on Lucene.NET

1 Answers