0
votes

We need to add 130 MB data into a searchable string field in the index,

However API has limit of 16 MB as mentioned into limit page listed below, besides, since its per call it;

"....Primitive collection example: If the document contains a field "Tags" of type Collection(Edm.String) with value ["budget"], and you execute a merge with value ["economy", "pool"] for "Tags", the final value of the "Tags" field will be ["economy", "pool"]. It will not be ["budget", "economy", "pool"]...."

Azure search limits

Question: Can I divide the data across multiple calls and upload full 130 MB data into a single index document? as that data belong to one record/entity on our index.

PS: merge at below link seems performing the replace instead of addition

Index Data Upload

1

1 Answers

1
votes

It is correct that the limit to a specific field for a single document in an Azure Cognitive Search index is 16MB. It is important to note however, that this is the size of the text that is put into the field and not of of the original document (if you are using documents). For example, you might have a 130 MB PDF, but the actual text is only a few MB which would fit.

As for going beyond that, I would be interested in why you might want to do that? In most cases, if there is actually this much text, it is typically more usable from a search perspective to split the text up into separate search index documents. For example, if the content was coming from a huge PDF, maybe you index one chapters such that one chapter represents a single document?