Azure Cognitive Search issues with fuzzy search and custom analyzers

Question

I'm working on implementing fuzzy search against some Azure search indexes. I'm seeing some inconsistencies that I can't quite figure out... Here's some background.

We have a few search indexes. It has been requested that we implement fuzzy search. Azure Cognitive search supports this feature, but I am seeing inconsistent behavior across search indexes.

For the sake of the example... Say we have index-a and index-b. Their schemas might look like this:

index-a

{
   entityId: string
   entityName: string
   entityNameSuggester: string
}

index-b

{
   entityId: string
   entityName: string
   entityNameSuggester: string
}

We have created a custom analyzer to allow for searches with special characters, and with contains behavior (/.*searchTerm.*/). Let's assume that the entityName field of each of these index schemas use this custom analyzer. The suggester fields are copies of another field, and are used for autocomplete.

I started my fuzzy search testing with index-a. A query like this would yield results:

{
    "search": "Bayer",
    "top": 25,
    "queryType": "full",
    "searchMode": "all",
    "count": true
}

If we turn it into a fuzzy search by appending a tilde to the search term, this query also yields results:

{
    "search": "Bayer~",
    "top": 25,
    "queryType": "full",
    "searchMode": "all",
    "count": true
}

So far, so good. However, we do fielded searches to reduce the scope of the search. The moment I specify search fields when doing a fuzzy search against index-a, I stop getting results:

{
    "search": "Bayer~",
    "searchFields": "entityName",
    "top": 25,
    "queryType": "full",
    "searchMode": "all",
    "count": true
}

This seems odd, and something got me wondering whether the custom analyzer applied to this field was somehow impacting the search.

So I tried running the same query (with different search terms) against index-b - in actuality, our index schemas are quite different from each other but each of them have the same custom analyzer applied to at least one field. My expectation was to get no results, but, somehow, doing a fielded fuzzy search against a field with the custom analyzer on index-b DID yield results.

Another weird thing is that, if I modify the fielded fuzzy search query against index-a to target the suggester field rather than the one with the custom analyzer applied, then the search yields results!

{
    "search": "Bayer~",
    "searchFields": "entityNameSuggester",
    "top": 25,
    "queryType": "full",
    "searchMode": "all",
    "count": true
}

The same is true for index-b, the difference being that this query yields results both for suggester and non-suggester fields on index-b, but ONLY for suggester fields on index-a.

I am not sure what to make of this. There's clearly something causing this to work on one index but not the other. If anyone is able to shed some light or hints into what the issue might be, I'd really appreciate it!

Michael Scott Michael Scott · Accepted Answer · 2021-08-02T16:38:04

Are you sure the field on which you are performing the fielded search has data that would satisfy your search criteria? If yes, I'd recommend deleting the custom analyzer and performing the same test again. Could your custom analyzer be the culprit here?

Azure Cognitive Search issues with fuzzy search and custom analyzers

1 Answers