we have an elastic search index with following mapping (showing only partial mapping relevant to this question)
"instFields": {
"properties": {
"_index": {
"type": "object"
},
"fieldValue": {
"fields": {
"raw": {
"index": "not_analyzed",
"type": "string"
}
},
"type": "string"
},
"sourceFieldId": {
"type": "integer"
}
},
"type": "nested"
}
as you can see fieldValue
type is string
: in original data in the database for that fieldValue
column is stored in a JSON
type column (in Postgresql). use case is such that when this data is stored fieldValue
can be valid JsNumber
, JsString
,JsBoolean
(any valid [JsValue][1]
now question is that when storing this fieldValue
in ES - it'll have to be a definite type - so we convert fieldValue
to String
while pushing data into ElasticSearch.
Following is a sample data from Elastic search
"instFields": [
{
"sourceFieldId": 1233,
"fieldValue": "Demo Logistics LLC"
},
{
"sourceFieldId": 1236,
"fieldValue": "169451"
}
]
this is where it gets interesting where now we want to run various metrics aggregations on fieldValue
- for e.g. if sourceFieldId = 1236
then run [avg][3]
on fieldValue
- problem is fieldValue
had to be stored as string
in ES - due to originally fieldValue
being JsValue
type field in the application. what's the best way to create mapping in elastic search such that fieldValue
can be stored with an appropriate type vs string
type so various metrics aggregation can be run of fieldValue
which are of type long
(though encoded as string
in ES)