Elasticsearch multiple search terms

Question

I've set an elastic index. I have 100,000 documents all with the following fields

{
"Make": "NISSAN",
"Model": "FUGA",
"Body Type": "SEDAN",
"Year of Manufacture": 2012,
"Country": "JAPAN",
"Fuel Type": "PETROL"
}

I need to create a search based on four possible terms

Make
Model
Year of Manufacture
Fuel Type

Below are four possible combinations for a search query

2012 nissan fuga petrol
nissan fuga 2012 petrol
petrol 2012 nissan fuga
nissan fuga petrol 2012

Assuming we have correct spelling on the search query, below is how i tried searching based on the search query

curl -X GET "localhost:9200/vehicles/_search?pretty" -H 'Content-Type: application/json' -d'
{
  "query": {
    "simple_query_string" : {
        "query": "2012 NISSAN FUGA PETROL",
        "fields": ["Make","Model","Year of Manufacture","Fuel Type"]
    }
  }
}

Out of surprise, the search returns an error below

    {
    "error": {
        "root_cause": [
            {
                "type": "query_shard_exception",
                "reason": "failed to create query: {\n  \"simple_query_string\" : {\n    \"query\" : \"2012 NISSAN FUGA PETROL\",\n    \"fields\" : [\n      \"Model^1.0\",\n      \"Make^1.0\",\n      \"Year of Manufacture^1.0\",\n      \"Fuel Type^1.0\"\n    ],\n    \"flags\" : -1,\n    \"default_operator\" : \"or\",\n    \"analyze_wildcard\" : false,\n    \"auto_generate_synonyms_phrase_query\" : true,\n    \"fuzzy_prefix_length\" : 0,\n    \"fuzzy_max_expansions\" : 50,\n    \"fuzzy_transpositions\" : true,\n    \"boost\" : 1.0\n  }\n}",
                "index_uuid": "3vd2zOgHRIq3BUAJ_EATVQ",
                "index": "vehicles"
            }
        ],
        "type": "search_phase_execution_exception",
        "reason": "all shards failed",
        "phase": "query",
        "grouped": true,
        "failed_shards": [
            {
                "shard": 0,
                "index": "vehicles",
                "node": "Xl_WpfXyTcuAi2uadgB4oA",
                "reason": {
                    "type": "query_shard_exception",
                    "reason": "failed to create query: {\n  \"simple_query_string\" : {\n    \"query\" : \"2012 NISSAN FUGA PETROL\",\n    \"fields\" : [\n      \"Model^1.0\",\n      \"Make^1.0\",\n      \"Year of Manufacture^1.0\",\n      \"Fuel Type^1.0\"\n    ],\n    \"flags\" : -1,\n    \"default_operator\" : \"or\",\n    \"analyze_wildcard\" : false,\n    \"auto_generate_synonyms_phrase_query\" : true,\n    \"fuzzy_prefix_length\" : 0,\n    \"fuzzy_max_expansions\" : 50,\n    \"fuzzy_transpositions\" : true,\n    \"boost\" : 1.0\n  }\n}",
                    "index_uuid": "3vd2zOgHRIq3BUAJ_EATVQ",
                    "index": "vehicles",
                    "caused_by": {
                        "type": "number_format_exception",
                        "reason": "For input string: \"NISSAN\""
                    }
                }
            }
        ]
    },
    "status": 400
}

Below is more information on my version of elastic

{
"name": "salim-HP-EliteBook-840-G5",
"cluster_name": "elasticsearch",
"cluster_uuid": "mSWKP4G1TSSq9rI3Hc0f6w",
"version": {
    "number": "7.5.1",
    "build_flavor": "default",
    "build_type": "tar",
    "build_hash": "3ae9ac9a93c95bd0cdc054951cf95d88e1e18d96",
    "build_date": "2019-12-16T22:57:37.835892Z",
    "build_snapshot": false,
    "lucene_version": "8.3.0",
    "minimum_wire_compatibility_version": "6.8.0",
    "minimum_index_compatibility_version": "6.0.0-beta1"
},
"tagline": "You Know, for Search"

}

Below is the index mapping for the vehicles index

    {
    "vehicles": {
        "mappings": {
            "_meta": {
                "created_by": "ml-file-data-visualizer"
            },
            "properties": {
                "Body Type": {
                "type": "keyword"
                },
                "Country": {
                    "type": "keyword"
                },
                "Fuel Type": {
                    "type": "keyword"
                },
                "Make": {
                    "type": "keyword"
                },
                "Model": {
                    "type": "text"
                },
                "Year of Manufacture": {
                    "type": "long"
                }
            }
        }
    }
}

How can i make a successful search based on my search criteria ?

Hi @salimsaid, I can't reproduce this experiment. Your query returns the expected document. Can you please share your index mapping ? — baitmbarek
@baitmbarek I have edited the question and added my index mapping. — salimsaid
In your mapping every property name is suffixed with a colon ':' is that intended ? — baitmbarek
@baitmbarek You were right about the extra colon : , i removed the colons and the query now returns number_format_exception , i guess elastic search is trying to parse Year of Manufacture and finds NISSAN instead of a number. How do i get around this one ? — salimsaid
You should try a multimatch query for this purpose : elastic.co/guide/en/elasticsearch/reference/current/… — baitmbarek

Assael Azran Assael Azran · Accepted Answer · 2019-12-22T12:03:04

UPDATE

cross_fields with synonym token filter.

A working example:

Mappings (Updated)

PUT my_index
{
  "mappings": {
    "properties": {
      "Make": {
        "type": "text"
      },
      "Model": {
        "type": "text"
      },
      "Body Type": {
        "type": "text"
      },
      "Year of Manufacture": {
        "type": "text",
        "fields": {
          "long": {
            "type": "long"
          }
        }
      },
      "Country": {
        "type": "text"
      },
      "Fuel Type": {
        "type": "text"
      }
    }
  },
  "settings": {
    "index": {
      "analysis": {
        "filter": {
          "my_syn_filt": {
            "type": "synonym",
            "synonyms": [
              "nisson,nissen => nissan",
              "foga => fuga"
            ]
          }
        },
        "analyzer": {
          "my_synonyms": {
            "filter": [
              "lowercase",
              "my_syn_filt"
            ],
            "tokenizer": "standard"
          }
        }
      }
    }
  }
}

Index few documents

PUT my_index/_doc/1
{
  "Make": "NISSAN",
  "Model": "FUGA",
  "Body Type": "SEDAN",
  "Year of Manufacture": 2012,
  "Country": "JAPAN",
  "Fuel Type": "PETROL"
}

PUT my_index/_doc/2
{
  "Make": "NISSAN",
  "Model": "FUGA",
  "Body Type": "SEDAN",
  "Year of Manufacture": 2013,
  "Country": "JAPAN",
  "Fuel Type": "PETROL"
}

PUT my_index/_doc/3
{
  "Make": "FIAT",
  "Model": "FUGA",
  "Body Type": "SEDAN",
  "Year of Manufacture": 2014,
  "Country": "JAPAN",
  "Fuel Type": "PETROL"
}

Search Query (Updated)

GET my_index/_search
{
  "query": {
    "multi_match": {
      "query": "NISSON FOGA 2012 PETROL",   ---> nisson and foga
      "fields": ["Make","Model","Year of Manufacture","Fuel Type"],
      "type": "cross_fields",
      "operator": "and",
      "analyzer": "my_synonyms"
    }
  }
}

Results

"hits" : [
  {
    "_index" : "my_index",
    "_type" : "_doc",
    "_id" : "1",
    "_score" : 1.2605431,
    "_source" : {
      "Make" : "NISSAN",
      "Model" : "FUGA",
      "Body Type" : "SEDAN",
      "Year of Manufacture" : 2012,
      "Country" : "JAPAN",
      "Fuel Type" : "PETROL"
    }
  }
]

Hope this helps

Elasticsearch multiple search terms

1 Answers