Jumps of Refresh Time on Elasticsearch Cluster

1
votes

I am trying to fight occasional spikes in Refresh Time on my logging service for API calls. Almost all of the requests to server are index/update pairs with refresh_interval -1. _refresh is called from cronjob every minute, but the spikes happen only 1-4 times per hour. The spikes are usually 6-8s.

Refresh Time

Translog durability is set to async. indices.memory.index_buffer_size: 50%

Or maybe it's just completely normal? I have no comparison, all my other projects do not index that many records. On average there are 170k index operations daily. I am not using bulk indexing, it's not possible in this scenario.

Thank you for all suggestions! Marcin

1
elasticsearch
Do you index in bulk and do you refresh the index while indexing ? - Christophe Roussy
Dear Christophe, To put it short: I refresh while indexing. Thank you, Marcin - mgo
For performance the best is usually to bulk import while indexing is off. elastic.co/guide/en/elasticsearch/reference/current/… and elastic.co/guide/en/elasticsearch/reference/current/… - Christophe Roussy
why you couldn't do bulk indexing? - Mysterion
I use ES for logging request/response to an external API. Each HTTP request consists of: 1. index call before external API call, 2. update call with response received. It all has to work within a single http request. It is not a perfect case for ES. - mgo

1 Answers