score synonyms equally in Elasticsearch

Question

Can we score the original string and synonyms equally in elasticsearch.

For eg. I created my synonyms file as:

pvt, private

ltd, limited

I created an index using synonym token filter. Then I indexed two documents:

curl -XPOST "http://localhost:9200/test1/test?pretty" -d 
    '{ "entityName" : "ABC International Pvt Ltd"}'

curl -XPOST "http://localhost:9200/test1/test?pretty" -d 
    '{ "entityName" : "ABC International Private Limited"}'

Now when I search "ABC International Pvt Ltd", it scores the first document as 1.15 and second document as 0.57.

Is there a way to treat the synonyms equally?

Created index using following settings:

curl -XPUT 'localhost:9200/test1?pretty' -H 'Content-Type: application/json' -d'
{
    "settings" : {
        "index" : {
            "analysis":{
                "analyzer":{
                    "my_analyzer":{
                        "tokenizer":"standard",
                        "filter":["asciifolding", "standard", "lowercase", "my_metaphone", "synonym"]
                    }
                },
                "filter":{
                    "my_metaphone":{
                        "type":"phonetic",
                        "encoder":"metaphone",
                        "replace":false
                    },
                    "synonym" : {
                      "type" : "synonym", 
                      "synonyms_path" : "synonyms.txt",
                      "ignore_case" : "true"
                    }
                }
            }
        }
    }
}'

Can you show how you defined the settings and mappings for your index? — Val
Is it possible you have a multi-shard index with very few documents? If so, try again with a single-shard index. Scoring happens at shard level, so you get weird results if you don't have a lot of documents. — dshockley

Rohit Patwa Rohit Patwa · Accepted Answer · 2017-10-24T15:55:02

Adding mapping while creating index did the job. Without the mapping, the synonym token filter was not even being applied. Below is the command I used to create index.

curl -XPUT 'localhost:9200/test1?pretty' -H 'Content-Type: application/json' -d' 
{
"settings" : {
  "analysis":{
    "filter":{
      "my_metaphone":{
        "type":"phonetic",
        "encoder":"metaphone",
        "replace":false
      },
      "synonym" : {
        "type" : "synonym", 
        "synonyms_path" : "synonym.txt",
        "ignore_case" : "true"
      }
    },
    "analyzer":{
      "my_analyzer":{
        "type":"custom",
        "tokenizer":"standard",
        "filter":["asciifolding", "standard", "lowercase", "my_metaphone", "synonym"]
      }
    }
  }
},
"mappings": {
  "test": {
    "properties": {
      "text": {
        "type": "text",
        "analyzer": "my_analyzer", 
        "search_analyzer": "my_analyzer" 
      }
    }
  }
}
}'

score synonyms equally in Elasticsearch

1 Answers