azure search performance issue

Question

I'm built an index in azure search. the index contains several fields but only 2 of the fields I marked as searchable. the index have full words pdf files from various sizes ( the text of them of course) and its purporse to be able to use in document search.

The index is large in size - 160k documents and 7.2 gb size.

When I do a query up until 2 letters everything is working as well but when I hit complex words the performance is bad.

for example: the phrase - elasti took 59 seconds.

In my query - i use filter,scoring profile highlight

after some investigation - i can see that the highlight feature is causing the problem.

Any idea? how can increase performance - maybe split into several fields in the index or something like that?

Answer 1

We deployed a change that improves highlighter performance this week. Reindexing is required to enable the feature. Please feel free to contact me directly at [email protected] if the issue persists. Thank you again for your time.

Nate

Answer 2

Your guess is right - search highlight is causing the issue here. When highlighting is enabled, Azure Search processes each item/text in the result set at query time to find text segments containing the seach hits. If each text is 4.5mb (7.2gb/160k), the process may take long. Regarding complex terms or phrases taking longer time, that would be because Azure Search stops processing the result after it finds enough number of segments. I work in Azure Search team. I will look at the code and confirm.