Elastic Search size to unlimited

Question

Am new to elastic search. Am facing a problem to write a search query returning all matched records in my collection. Following is my query to search record

{
   "size":"total no of record" // Here i need to get total no of records in collection
   "query": {
      "match": {
         "first_name": "vineeth"
      }
   }
}

By running this query i am only getting maximum 10 records, am sure there is more than 10 matching records in my collection. I searched a lot and finally got size parameter in query. But in my case i dont know the total count of records. I think giving an unlimited number to size variable is not a good practice, so how to manage this situation please help me to solve this issue, Thanks

Answer 1

It's not very common to display all results, but rather use fromand size to specify a range of results to fetch. So your query (for fetching the first 10 results) should look something like this:

{
    "from": 0,
    "size": 10,
    "query": {
        "match": {
            "first_name": "vineeth"
        }
    }
}

This should work better than setting size to a ridiculously large value. To check how many documents matched your query you can get the hits.total (total number of hits) from the response.

Answer 2

To fetch all the records you can also use scroll concept.. It's like cursor in db's..
If you use scroll, you can get the docs batch by batch.. It will reduce high cpu usage and also memory usage..

For more info refer

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-scroll.html

Answer 3

To get all records, per de doc, you should use scroll. Here is the doc: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-scroll.html

But the idea is to specify your search and indicate that you want to scroll it:

curl -XGET 'localhost:9200/twitter/tweet/_search?scroll=1m' -d '
{
   "query": {
    "match" : {
        "title" : "elasticsearch"
     }
 }
}'

in the scroll param you specify how long you want the search results available.

Then you can retrieve them with the returned scroll_id and the scroll api.

Answer 4

in new versions of elastic (e.g. 7.X), it is better to use pagination than scroll (deprecated): https://www.elastic.co/guide/en/elasticsearch/reference/current/paginate-search-results.html

deprecated in 7.0.0:

GET /_search/scroll/<scroll_id>