NEST Sort by Text field

Question

I am trying to write a search module that uses NEST and keeps the search functionality within its own bounded context.

To achieve this, I have the following user profile search:

public class UserProfileSearch : IUserProfileSearch
{
    ...

    public async Task<PagedItems<UserProfileModel>> FindAsync(string searchTerm, int skip, int take)
    {
        var client = _elasticClientProvider.GetClient();
        var response = await client.SearchAsync<ElasticUserProfileModel>(s => s
            .Index(_elasticConfiguration.GetIndex())
            .From(skip).Size(take)
            .Query(q => q.MultiMatch(m => m.Fields(f => f
                .Field(u => u.Email)
                .Field(u => u.FirstName)
                .Field(u => u.LastName))
                .Query(searchTerm)))
            .Sort(q => q.Ascending(u => u.Email)));
        var count = await client.CountAsync<ElasticUserProfileModel>(s => s.Index(_elasticConfiguration.GetIndex()));
        return new PagedItems<UserProfileModel> { Items = response.Documents.Cast<UserProfileModel>().ToArray(), Total = count.Count };
    }
}

The response is failing consistently with this report:

{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [email] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"xxxx","node":"xxxx","reason":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [email] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}}]},"status":400}

However, I have done what was recommended in the report, but the same error keeps happening. I have defined

public class ElasticUserProfileModel : UserProfileModel
{
    [Text(Fielddata = true)] public override string Email { get => base.Email; set => base.Email = value; }
}

which should be exactly what the report is asking for. I am rebuilding the index with the ElasticUserProfileModel during each end-to-end test.

I have also tried using the Keyword attribute rather than the Text attribute, but that is generating exactly the same error.

If I sort by Id (which is numeric) instead of Email, there is no error. But this is a significantly less useful search.

Is there a simple way to fix this?

What does the mapping look like? Is the mapping picking up the attribute on the overridden property as expected? — Russ Cam
@RussCam -- yes it is. The descriptor shows the Email property as having FieldData set to true. — Rob Lyndon

Rob Lyndon Rob Lyndon · Accepted Answer · 2017-10-30T12:04:24

Based on the documentation at https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/multi-fields.html, I discovered that a POCO string is automatically mapped to both a keyword and a text field. The Suffix() extension method was all I needed to enable string sorting to work.

I deleted the ElasticUserProfileModel derived class, and the FindAsync() method became

public async Task<PagedItems<UserProfileModel>> FindAsync(string searchTerm, int skip, int take)
{
    var client = _elasticClientProvider.GetClient();
    var response = await client.SearchAsync<UserProfileModel>(s => s
        .Index(_elasticConfiguration.GetIndex())
        .From(skip).Size(take)
        .Query(q => q.MultiMatch(m => m.Fields(f => f
            .Field(u => u.Email)
            .Field(u => u.FirstName)
            .Field(u => u.LastName))
            .Query(searchTerm)))
        .Sort(q => q.Ascending(u => u.Email.Suffix("keyword"))));
    var count = await client.CountAsync<UserProfileModel>(s => s.Index(_elasticConfiguration.GetIndex()));
    return new PagedItems<UserProfileModel> { Items = response.Documents.ToArray(), Total = count.Count };
}

which solved the problem.

NEST Sort by Text field

1 Answers