filter and return only specific fields in elasticsearch

Question

I'm trying to create a request where I can extract a specific metric from Elasticsearch, so I could manage data faster. I got rid of metadata and unnecessary data with a request as follow :

get localhost:9200/metricbeat-7.12.0/_search?size=2000&pretty&filter_path=hits.hits._source
{
    "_source": ["@timestamp", "labels","prometheus"]
}

I get something like that which is closer to what I want. Now I would like an additional filter where I get only the metrics "prometheus.metrics.windows_cpu_time_total" and not the other metrics.

   {
    "hits": {
        "hits": [
            {
            {
                "_source": {
                    "@timestamp": "2021-04-29T15:35:57.518Z",
                    "prometheus": {
                        "metrics": {
                            "windows_service_status": 0
                        },
                        "labels": {
                            "instance": "localhost:9182",
                            "name": "timebrokersvc",
                            "job": "prometheus",
                            "status": "lost comm"
                        }
                    }
                }
            },
            {
                "_source": {
                    "@timestamp": "2021-04-29T15:35:57.518Z",
                    "prometheus": {
                        "metrics": {
                            "windows_cpu_time_total": 29480.625
                        },
                        "labels": {
                            "mode": "idle",
                            "core": "0,0",
                            "instance": "localhost:9182",
                            "job": "prometheus"
                        }
                    }
                }
            }}]}}

I tried a field search which doesn't seem to work as well. Could someone point me to what's going wrong with my queries?

{
  "query": {
    "match_all": {}
  },
    "fields": [
    "prometheus.metrics", 
    {
        "field": "windows_cpu_time_total"
    }]
  }

Thank you in advance

Answer 1

@ESCoder I couldn't post a comment anymore. I'll answer your question here. Here is the overview of the mapping with Prometheus properties included.

"prometheus": {
                "properties": {
                    "*": {
                        "properties": {
                            "counter": {
                                "type": "object"
                            },
                            "histogram": {
                                "type": "object"
                            },
                            "rate": {
                                "type": "object"
                            },
                            "value": {
                                "type": "object"
                            }
                        }
                    "metrics": {
                            "properties": {
                        "*": {
                            "type": "object"
                        },
                        "windows_cpu_time_total": {
                            "type": "double"
                        },}}

Answer 2

In elasticsearch version 7.10 and 7.11, fields functionality was in beta version, as stated in the official documentataion

But in elasticsearch version 7.12, the fields option, is working fine

Adding a working example with index data, search query, and search result

Index Data:

{
  "@timestamp": "2021-04-29T15:35:57.518Z",
  "prometheus": {
    "metrics": {
      "windows_cpu_time_total": 29480.625
    },
    "labels": {
      "mode": "idle",
      "core": "0,0",
      "instance": "localhost:9182",
      "job": "prometheus"
    }
  }
}

Search Query:

POST _search?size=2000&pretty&filter_path=hits.hits
{
  "query": {
   "match_all": {}
  },
  "fields": [
    "prometheus.metrics.windows_cpu_time_total"
  ],
  "_source": false
}

Search Result:

{
  "hits": {
    "hits": [
      {
        "_index": "67588900",
        "_type": "_doc",
        "_id": "1",
        "_score": 1.0,
        "fields": {
          "prometheus.metrics.windows_cpu_time_total": [
            29480.625
          ]
        }
      }
    ]
  }
}

Here an overview of the index mapping

"prometheus": {
                "properties": {
                    "*": {
                        "properties": {
                            "counter": {
                                "type": "object"
                            },
                            "histogram": {
                                "type": "object"
                            },
                            "rate": {
                                "type": "object"
                            },
                            "value": {
                                "type": "object"
                            }
                        }
                    "metrics": {
                            "properties": {
                        "*": {
                            "type": "object"
                        },
                        "windows_cpu_time_total": {
                            "type": "double"
                        },}}