Solr fulltext search in production - which is better sunspot:solr bundled solution or standalone tomcat+solr?

Question

We are utilizing sunspot's Solr gem for a while in development mode. The application is now needs to be moved to production environment.

Are there any cons of setting up solr in production environment using the bundled sunspot gem ?

I generally like tomcat + solr idea but it's more complex to set up.

Sunspot bundled gem is easy. Provided that we have 1 rails project in 3 stages (staging,production,QA), 2 app servers and one db/solr server one has to :

1. push rails project to db server
2. launch solr on db server in production environment with bundle exec rake sunspot:solr:start for (we just use different port for each solr instance)
3. reindex with rake sunspot:solr:reindex
4. modify sunspot.yml for app server to make them connect to db server on specific stage port

Above is applicable for X instances of production/staging solrs.

Tomcat6 + Sunspot is not so easy and time/result effective. As I understand (fix me) one has to :

1. Install and maintain tomcat on db server
2. Provided that we want to have 3 solr instances - deploy 3 solr wars with separate solrs (or use the multicore solr which is quite difficult to set up)
3. deploy the rails project onto the db server
4. Everytime we change our schema/configuration/whatever in rails - we have to take care of generating solr xmls and confs and copy it to tomcat webapps
5. Modify the directories in solr conf/xml so it points to index files etc.
6. Reload tomcat solr application instance on every rails deploy what includes some scripting and "non-rails" approach to deployment procedure :)

Answer 1

I did something similar lately and had to answer the same questions. My problem space looked like having two languages (so preferably two cores and maybe more for other search options in future), having to index data over several table and search/count facets, partly in hierarchical structures like product groups and hierarchical tags like car_brand/model/type/build_year.

As far as I have seen sunspot works best (only) if you have relative 'flat' data, at best simply have to define for a single model which fields to use. The moment you need to combine several models into a single document for facets, it becomes difficult. (But since I didn't use Sunspot I can't really tell for sure). You can (and should) still use the rsolr gem for the communication with Solr. But if you use Sunspot in development, it is most likely working for you anyway.

For your details about the tomcat thing:

1. Install and maintain tomcat on db server

yes, some work but can be handled

1. Provided that we want to have 3 solr instances - deploy 3 solr wars with separate solrs (or use the multicore solr which is quite difficult to set up)

I used multi core. It's not really difficult to set up. You are better off learning a bit about Solr configuration anyway. Largest part of multi core is just having two sub directories that contain essentially the same set of XML config files.

1. deploy the rails project onto the db server

have to do this anyway :)

1. Everytime we change our schema/configuration/whatever in rails - we have to take care of generating solr xmls and confs and copy it to tomcat webapps

Yes, write a few scripts. Alternately you could have a git repo that contains the tomcat/solr folders and Rails project folder or symlink those folders. I haven't found a good solution yet and have to copy a bit stuff once in a while.

1. Modify the directories in solr conf/xml so it points to index files etc.

yep, a few symlinks can help to keep things easy

1. Reload tomcat solr application instance on every rails deploy what includes some scripting and "non-rails" approach to deployment procedure :)

Only if the deploy actually changes anything related to your search. Have a few scripts. There are Rails deployment solutions that would help, but as I do not mind to do a few things manually I didn't bother to install.

All in all I think I have more control and insight into the configuration of Solr and can better use some of it's more complex features like facets with hierarchies. Actually Solr may look a bit complicated at first, but after some time you get into it and then it becomes a great tool.