Here is my sample document, I am indexing similar documents with help of POST /user_data1/_doc/ in kibana.
{
"id": 15,
"name": "abcd",
"source": "csv_status",
"profile_complition": "70%",
"creation_date": "2020-04-02",
"current_position": [
{
"position": "Financial Reporting",
"position_category": "Finance",
"position_level": 2
}
],
"seeking_position": [
{
"position": "Financial Planning and Analysis",
"position_category": "Finance",
"position_level": 3
}
],
"last_updation_date": "2021-02-02",
"experience": [
{
"brand": "Other",
"company": "Other-Apartments and Residences",
"brand_segment": "Luxury",
"property_type": "All-Inclusive",
"duration": "2 years",
"real_estate_type": "Institutional"
},
{
"brand": "Accor",
"company": "Accor LLC",
"brand_segment": "Luxury",
"property_type": "Condo",
"duration": "2 years",
"real_estate_type": "Family Office"
},
{
"brand": "SO",
"company": "Accor LLC",
"brand_segment": "Luxury",
"property_type": "Condo",
"duration": "2 years",
"real_estate_type": "Family Office"
},
{
"brand": "Other",
"company": "Other-Multi-Family",
"brand_segment": "Independent",
"property_type": "Convention",
"duration": "2 years",
"real_estate_type": "Family Office"
},
{
"brand": "Other Lifestyle – Luxury",
"company": "Other Lifestyle – Luxury",
"brand_segment": "Extended Stay",
"property_type": "Condo",
"duration": "2 years",
"real_estate_type": "Family Office"
}
]
}
Now in document under 'experience' key 'brand' is the sub-group of main group 'company'. As you can see in doc that 'brand' may have similar entries like 'other', but is separated by 'company' key. Similar structure can be present both within and across document. I intend to aggregate the document based on both 'brand' and 'company'. The query I tried is-
GET user_data1/_search
{
"size": 0,
"aggs": {
"nested_aggs": {
"nested": {
"path": "experience"
},
"aggs": {
"by_brand": {
"terms": {
"field": "experience.brand"
},
"aggs": {
"by_company": {
"terms": {
"field": "experience.company"
}
}
}
}
}
}
}
}
For which I am getting error as-
{
"error" : {
"root_cause" : [
{
"type" : "aggregation_execution_exception",
"reason" : "[nested] nested path [experience] is not nested"
}
],
"type" : "search_phase_execution_exception",
"reason" : "all shards failed",
"phase" : "query",
"grouped" : true,
"failed_shards" : [
{
"shard" : 0,
"index" : "user_data1",
"node" : "aa0oyCw6RLa2J5aDALunsA",
"reason" : {
"type" : "aggregation_execution_exception",
"reason" : "[nested] nested path [experience] is not nested"
}
}
]
},
"status" : 500
}
Here is the mapping of my document-
{
"user_data1" : {
"aliases" : { },
"mappings" : {
"properties" : {
"creation_date" : {
"type" : "date"
},
"current_position" : {
"properties" : {
"position" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"position_category" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"position_level" : {
"type" : "long"
}
}
},
"experience" : {
"properties" : {
"brand" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"brand_segment" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"company" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"duration" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"property_type" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"real_estate_type" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"id" : {
"type" : "long"
},
"last_updation_date" : {
"type" : "date"
},
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"profile_complition" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"seeking_position" : {
"properties" : {
"position" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"position_category" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"position_level" : {
"type" : "long"
}
}
},
"source" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"settings" : {
"index" : {
"routing" : {
"allocation" : {
"include" : {
"_tier_preference" : "data_content"
}
}
},
"number_of_shards" : "1",
"provided_name" : "user_data1",
"creation_date" : "1615227708037",
"number_of_replicas" : "1",
"uuid" : "8B3dnDKKQjmDNkjAksduJQ",
"version" : {
"created" : "7100299"
}
}
}
}
}
I understand that my experience field is not nested, and may be that's the reason of this error. I am relatively new to elasticsearch and hence finding difficult to make my document nested type while indexing. I tried making it nested by mapping and then populating the document, but error remains same. Help me to
- make desired fields nested at time of indexing.
- To write composite aggregation that bucket aggregates on basis of 'brand' and 'country'.
Let me know if further clarification required
