What is the purpose of BoolQuery's "filter" in ElasticSearch?

Question

I read the documentation of BoolQuery and according to it, this the purpose,

filter

The clause (query) must appear in matching documents. However unlike must the score of the query will be ignored. Filter clauses are executed in filter context, meaning that scoring is ignored and clauses are considered for caching.

Also from BoolQueryBuilder class:

   /**
     * Adds a query that <b>must</b> appear in the matching documents but will
     * not contribute to scoring. No {@code null} value allowed.
     */
    public BoolQueryBuilder filter(QueryBuilder queryBuilder) {
        if (queryBuilder == null) {
            throw new IllegalArgumentException("inner bool query clause cannot be null");
        }
        filterClauses.add(queryBuilder);
        return this;
    }

but I can't get my head around, this. When should I use filter vs (should or must)

Here is the example I am working on :

I want to filter out some records based on the following assumptions :

Fetch All

1) Records where deleted=0 and isPrivate=true

AND

2) Records where (isPrivate=false or [isPrivate=true and createdBy=loggedInUser])

Here are the 2 queries which give the same result, I want to know what filter query signifies

Result without Filter using just must and should clause.

"query": {
    "bool": {
      "must": [
        {
          "term": {
            "deleted": {
              "value": "0",
              "boost": 1
            }
          }
        },
        {
          "match": {
            "isPrivate": {
              "query": true
            }
          }
        },
        {
          "bool": {
            "should": [
              {
                "term": {
                  "isPrivate": {
                    "value": "false",
                    "boost": 1
                  }
                }
              },
              {
                "bool": {
                  "must": [
                    {
                      "term": {
                        "createdBy": {
                          "value": "1742991596",
                          "boost": 1
                        }
                      }
                    },
                    {
                      "term": {
                        "isPrivate": {
                          "value": "true",
                          "boost": 1
                        }
                      }
                    }
                  ],
                  "adjust_pure_negative": true,
                  "boost": 1
                }
              }
            ]
          }
        }
      ]
    }
  },

Query with using filter

"query": {
    "bool": {
      "adjust_pure_negative": true,
      "boost": 1,
      "filter": [
        {
          "bool": {
            "must": [
              {
                "term": {
                  "deleted": {
                    "value": "0",
                    "boost": 1
                  }
                }
              },
              {
                "match": {
                  "isPrivate": {
                    "query": true
                  }
                }
              }
            ],
            "should": [
              {
                "term": {
                  "isPrivate": {
                    "value": "false",
                    "boost": 1
                  }
                }
              },
              {
                "bool": {
                  "must": [
                    {
                      "term": {
                        "createdBy": {
                          "value": "1742991596",
                          "boost": 1
                        }
                      }
                    },
                    {
                      "term": {
                        "isPrivate": {
                          "value": "true",
                          "boost": 1
                        }
                      }
                    }
                  ],
                  "adjust_pure_negative": true,
                  "boost": 1
                }
              }
            ],
            "adjust_pure_negative": true,
            "boost": 1
          }
        }
      ]
    }
  }

Val Val · Accepted Answer · 2019-06-28T05:05:42

In your case, you should definitely use bool/filter since you don't have any constraint that contributes to scoring, all constraints are yes/no matches, and by using filter you can benefit from filter caches (which you don't when using must)

So definitely go with the filter option, but with a slight modification (you don't really need must at all and your boolean logic is not properly translated to bool queries):

{
  "query": {
    "bool": {
      "minimum_should_match": 1,
      "filter": [
        {
          "term": {
            "deleted": {
              "value": "0",
              "boost": 1
            }
          }
        },
        {
          "bool": {
            "minimum_should_match": 1,
            "should": [
              {
                "term": {
                  "isPrivate": {
                    "value": "false",
                    "boost": 1
                  }
                }
              },
              {
                "bool": {
                  "filter": [
                    {
                      "term": {
                        "createdBy": {
                          "value": "1742991596",
                          "boost": 1
                        }
                      }
                    },
                    {
                      "term": {
                        "isPrivate": {
                          "value": "true",
                          "boost": 1
                        }
                      }
                    }
                  ]
                }
              }
            ]
          }
        }
      ]
    }
  }
}

So to sum up:

should = OR condition
must = AND condition (when scoring is desired)
filter = AND condition (when scoring is not desired and/or when you want to benefit from filter caching)
Bonus: must_not = NOT condition

What is the purpose of BoolQuery's "filter" in ElasticSearch?

1 Answers