elasticsearch stops indexing new documents after a while, using Tire

Question

I have my website running, with ElasticSearch, using Tire.

In the background, I have a rake task that is looping over thousands of objects and creating one record for each of them.

At first, the records are indexed, and every time I refresh my website, I can see the number of records found go up. But at some point, it just stops. It saves the records to mysql, but does not index them anymore. Note that the site and the search are still up and running.

Why is that? Is there some kind of lock or...?

edit: Here are errors I see in elasticsearch.log

[2013-03-16 08:49:34,344][WARN ][netty.channel.socket.nio.AbstractNioSelector] Failed to accept a connection. java.io.IOException: Too many open files

[Thakos] [maps][2] failed to read latest segment infos on flush java.io.FileNotFoundException: /usr/local/share/elasticsearch/data/elasticsearch/nodes/0/indices/maps/2/index/segments_3 (Too many open files)

So I guess that's need, I need to look at how to deal with this error.

did you look at your rake, rails, database, elasticsearch logs? — phoet
Thanks. It was stupid of me not to look into elasticsearch logs... You can add an answer if you want about having to increase the number of open file descriptors available, as suggested on the official website: elasticsearch.org/guide/reference/setup/installation.html and I'll accept it. — Robin

Robin Robin · Accepted Answer · 2013-03-18T02:57:13

I basically missed an important part of the "Installation guide": http://www.elasticsearch.org/guide/reference/setup/installation.html

I only had like 1024 open file descriptors allowed for the user running elasticsearch, and I needed around 32k (max set to 64k).

With these settings, everything's running perfectly.

elasticsearch stops indexing new documents after a while, using Tire

1 Answers