Stemming using Tire library

Question

I am performing some basic search functions using ElasticSearch and Tire but the basic configuration of the snowball stemming analyzer has me stumped. I'm pretty much following the code example from the GitHub page: https://github.com/karmi/tire

Here's a Ruby sample file (Ruby 1.9.3, Tire 1.8.25):

require 'tire'

Tire.index 'videos' do
  delete
  create :mappings => {
  :video => {
      :properties => {
        :code                => { :type => 'string' },
        :description         => { :type => 'string', :analyzer => 'snowball' }
      }
  }
}
end

videos = [
    { :code => '1', :description => "some fight video" },
    { :code => '2', :description => "a fighting video" }
]

Tire.index 'videos' do
    import videos
    refresh
end

s = Tire.search 'videos' do
   query do
      string 'description:fight'
   end
end

s.results.each do |document|
   puts "* #{document.code} - #{document.description}"
end

I would have expected this to yield both records in the matches because fight and fighting have the same stem. However, it only returns the first record:

* 1 - some fight video

This would indicate that the default analyzer is being used rather than the one I'm configuring.

I am aware of passing the actual field in the query string per this question (ElasticSearch mapping doesn't work) and have successfully run this code so my ElasticSearch installation seems fine.

What do I need to change for Tire to return both records for this query (ie how do I get stemming working here)?

concept47 concept47 · Accepted Answer · 2013-05-20T21:16:17

I would have expected this to yield both records in the matches because fight and fighting have the same stem. However, it only returns the first record:

right. 'fight' stems to 'fight' and returns the result with only "fight" in it. Fighting will do exactly the same thing, unless you set up your search index to match otherwise.

If you want it to behave the way you describe, you'll probably want to make your default index use an edge ngram analyzer so that "fight" will also match "fighting" and return it. This will also have, what I think is the desirable effect, of matching both 'fight' and 'fighting' if you query for "fighting" too.

Stemming using Tire library

2 Answers