Mix DSL and URI query in Elasticsearch

Question

I'm trying to write a python class that takes a properly formatted Elasticsearch JSON query as a parameter. So far so good.. however, as part of that class I'd like to also take a "to" and "from" parameter to limit the date range that the query runs over.

Is there a way to combine the JSON for the DSL query along with URI parameters to pass in the date and time constraint?

I know I can limit the time using a range parameter like this:

GET /my-awesome-index*/_search
{
    "query": 
    {
        "bool": 
        {
            "must": [{"match_all": {}}],
            "filter": 
            [
                {"range": {"date_time": {"gte": "now-24h","lte": "now"}}},
                {"match_phrase": {"super_cool_field": "foo"}},
                {
                    "bool": 
                    {
                        "should": 
                        [
                            {"match_phrase": {"somewhat_cool_field_1": "bar"}},
                            {"match_phrase": {"somewhat_cool_field_2": "boo-ta"}}
                        ],
                        "minimum_should_match": 1
                    }
                }
            ]
        }
    }
}

and that's all well and good.. but, I want to craft my class to make the timeframe a variable. I also know I can limit the timeframe by submitting the URL like this..

GET /my-awesome-index*/_search?q=date_time:[1611732033412+TO+1611777796000]
{
    "query": 
    {
        "bool": 
        {
            "must": [{"match_all": {}}],
            "filter": 
            [
                {"match_phrase": {"super_cool_field": "foo"}},
                {
                    "bool": 
                    {
                        "should": 
                        [
                            {"match_phrase": {"somewhat_cool_field_1": "bar"}},
                            {"match_phrase": {"somewhat_cool_field_2": "boo-ta"}}
                        ],
                        "minimum_should_match": 1
                    }
                }
            ]
        }
    }
}

However, when I submit that Elasticsearch only seems to consider the timeframe from the URI and ignores the DSL JSON entirely.

Is there a way to get Elastic to consider / concatenate the two queries into one?

I'm considering programmatically making the range query something like this

range_part = '{{"range":{{"{}":{{"gte":"{}","lte":"{}"}}}}}}'.format(field,start,end)

And then dynamically inserting into any JSON the class takes.. but that seems cumbersome since there are so many formats available for the query and finding where to put the string etc..

Joe - Elasticsearch Handbook Joe - Elasticsearch Handbook · Accepted Answer · 2021-01-27T22:25:39

Your suspicion regarding q is right. Quoting from the docs:

The q parameter overrides the query parameter in the request body. If both parameters are specified, documents matching the query request body parameter are not returned.

So q and query cannot be combined.

As to "programatically making the range query" -- since you don't know in what format you'll receive the queries, the standard approach would be to traverse the query json, find/create the correct bool-must/bool-filter and set the range query there.

But let's take a step back -- maybe your class shouldn't be expecting a pre-baked JSON query in the first place. Why not pass a query config array like

[
  {
    "type": "match_phrase",
    "field": "super_cool_field",
    "value": "foo"
  },
  ...
]

and build the query yourself? That way you have full control over what gets passed downstream to ES. Plus, adding a date range query would be piece of cake...

Mix DSL and URI query in Elasticsearch

1 Answers