not_analyzed is not working as expected

Question

Mapping:

include Tire::Model::Search
mapping do
  indexes :name, :boost => 10
  indexes :account_id
  indexes :company_name
  indexes :email, :index => :not_analyzed
end

def to_indexed_json
  to_json( :only => [:name, :account_id, :email, :company_name], 
         )
end

From the above mapping the it can be seen that the email field is set to not_analyzed (no broken tokens). I have an user with email [email protected].

Now when I search for vamsikrishna, the result is showing the user...I guess it is using the default analyzer. why?

But, it should be shown only when the complete email is specified I guess ([email protected]). Why is the :not_analyzed not considered in this case? Please help.

I need only the email field to be set as not_analyzed, other fields should use standard analyzer (which is done by default).

Answer 1

You are searching using the _all field. It means that you are using analyzer specified for _all, not for email. Because of this the analyzer specified for email doesn't affect your search.

There are a couple of ways to solve this issue. First, you can modify the analyzer for _all field to treat emails differently. For, example you can switch to uax_url_email tokenizer that works as standard tokenizer, but doesn't split emails into tokens.

curl -XPUT 'http://localhost:9200/test-idx' -d '{
  "settings" : {
    "index": {
      "analysis" :{
        "analyzer": {
          "default": {
            "type" : "custom",
            "tokenizer" : "uax_url_email",
            "filter" : ["standard", "lowercase", "stop"]
          }
        }
      }
    }
  }
}
'

The second way is to exclude email field from _all and use your query to search against both fields at the same time.

Answer 2

try :analyzer => 'keyword' instead of :index => :not_analyzed

what it does is to tokenize the string and hence it will be searchable only as a whole. Dont forget to reindex !

Ref - http://www.elasticsearch.org/guide/reference/index-modules/analysis/keyword-analyzer.html

And still, if u are getting results by searching for vamsikrishna, check if you have other searchable fields with same value (for eg, name / company)

Answer 3

You're right, you should search for the whole field content in order to have a match on it if the specific field is not analyzed.

There are two options:

• The mapping hasn't been submitted correctly. You can check your current mapping through the get mapping api: 'localhost:9200/_mapping' will give you the mapping of all your indexes. Not a tire expert, but shouldn't you provide not_analyzed as a string? 'not_analyzed' instead of :not_analyzed?
• If you see that your mapping is there, that means you are searching on some other fields that match. Are you specifying the name of the field in your query?