ElasticSearch: when to use multi-field

Question

We have an index with a keyword field that is very often an ip address, but not always. We'd like to be able to search this index on that field using not just keywords but also CIDR notation, which is supported only for fields of type 'ip'. On the surface, this looks like a use case for multi-fields.

From https://www.elastic.co/guide/en/elasticsearch/reference/current/multi-fields.html:

It is often useful to index the same field in different ways for different purposes. This is the purpose of multi-fields

So it seems like the following mapping would make sense for us:

{
    "mappings": {
        "my_field": {
            "type": "keyword"
            "fields": {
                "ip": {
                    "type": "ip"
                    "ignore_malformed": true
                }  
            }
        }
    }
}

So, when our application has a set of non-ip addresses, ip addresses, and CIDR-notation blocks/ranges of ip addresses and needs to query by them, I assume the application would split that set into one set with non-ip addresses and another with ip addresses/CIDR-notation blocks and make two separate terms filters from them in my query, like so:

{
  "query": {
    "bool": {
      "filter": [
        {
          "terms": {
            "my_field.ip": [
              "123.123.123.0/24",
              "192.168.0.1",
              "192.168.16.255",
              "192.169.1.0/24"
            ]
          }
        },
        {
          "terms": {
            "my_field": [
              "someDomain.com",
              "notAnIp.net"
            ]
          }
        }
      ]
    }
  }
}

Is this a proper use of multi-fields? Should we be achieving this some other way? It's unlike the examples given for using multi-fields in that it's really a subset of the values for the field, not all, because I'm using ignore_malformed to discard the non-ip addresses from the sub-field. If there's a better way, what is it?

In the process of testing but it seems like it should work, thanks! — b15
Looks like I'm incorrectly using two filters that I want to 'or'' together, but this is only returning records that match both. — b15

user156327 user156327 · Accepted Answer · 2020-11-25T01:35:35

Yes, your understanding of multi-fields is correct, you just need to understand that you need to explicitly define the sub-field definition(data-type and analyzer) and also map them explicitly so that it uses the defined(data-type and analyzer).

Now once data is indexed in the format you wanted, you can include/exclude the sub-fields based on your use-case.

Multi-fields with multiple analyzers which is very common to implement multi-lingual search is a better example which you can refer.

ElasticSearch: when to use multi-field

1 Answers