How can I access script fields in queries made with NEST on Elasticsearch?

Question

I have a POCO that has a coordinate field, and the mapping and geodistance search with NEST all works as expected. However, what I'm trying to do with NEST is to also return the distance from the point specified in the GeoDistance query as part of the search results. My research thus far indicates that there are two ways to do this:

request a sort on the location
create a script field as part of the query that runs the distance calculation inline

I decided to run with option 2, and here's what some of the code looks like:

public class User
{
    public int ID { get; set; }
    [GeoPoint]
    public Coordinate Location { get; set; }
}

And the query itself:

var results = elasticClient.Search<User>(s => s
    .Index("user")
    .Query(q => q
        .GeoDistance(gd => gd
           .Field(f => f.Location)
           .Distance(Distance.Miles(distance))
           .Location(GeoLocation.TryCreate(data.Latitude, data.Longitude))
        ) && q
        .Exists(e => e
           .Field("location")
        )
    )       
    .ScriptFields(sf => sf
        .ScriptField("distance", sf2 => sf2
            .Inline(String.Format("doc['location'].arcDistance({0},{1})", data.Latitude, data.Longitude))    
            .Lang("painless")
        )
    )                
);

While running this query directly against elasticsearch seems fine, when I do it NEST I have a few issues:

I cannot access the "distance" script field I defined in my POCO object unless I add it to the definition
If I create a subclass for the User POCO, ElasticSearch does not deserialize the fields defined in the parent class (which means I only get the Distance script field, not the ID or Location fields, and the Documents array objects are all Null)
I'm trying to keep a separation between the actual POCO I use for mapping, and the search result object, but I don't know how to mark an attribute / field so that NEST won't map it, but will deserialize the result of the script field into the property

My question is: what is the recommended way to access script fields in NEST? Do I have to forego auto-mapping entirely? Or should I just go for the sort option even though I don't particularly want to sort the results?

I think someone had a similar query - github.com/elastic/elasticsearch-net/issues/432 - but I don't think the proposed solution of being able to access the raw fields on the Hit object was ever implemented? — Henry C

Henry C Henry C · Accepted Answer · 2017-08-10T15:02:28

I eventually managed to get close to what I wanted by:

explicitly telling my query to include the _source stored fields - it seems that if you use any script fields you need to do this or _source / documents will not be returned.
adding the [ElasticsearchType(Name = "User")] attribute to my Search Result subclass
when enumerating through the Hits in the result, accessing my script field:
```
((JArray)item.Fields["distance"]).Value<double>(0) / 1609.344 
```

If anyone has a cleaner way of doing this, please do let me know!

How can I access script fields in queries made with NEST on Elasticsearch?

2 Answers