In our application, we receive batches of files that need to be preprocessed and loaded into MarkLogic.
To do this, we need to:
- Load the files in a temporary MarkLogic working directory
- Preprocess (node operations on the
XMLfile) - Move documents to their intended destination MarkLogic directory using
xdmp:document-insert
While doing (3), we get expanded tree cache error, for a batch of 1500 docs (upto 400 docs, it works ok. Any higher number, the error pops up).
The algorithm steps for our code:
Get total number of docs in working directory = totalRec
for Ctr = 1 to totalRec
Get specific node values for current doc
Frame the target URI where doc is to be loaded
Insert document using xdmp:document-insert
We even tried using transaction begin/commits within the for loop but nothing seems to work. Any thoughts on how to fix this issue?