XML file upload using MLCP

Question

We are trying to upload xml files(some of them are of 2GB) but they are not getting uploaded in database using MLCP.

I created a new database and forest and new port .

Made changes to mlcp.bat as below

set OPTFILE="load_mlcp.txt"
call d:\mlcp-1.3-3-bin\mlcp-1.3-3\bin\mlcp.bat -options_file %OPTFILE% 
echo "Data Load Complete"
pause

The Load_mlcp.txt file has below code

import
-mode
local
-host
localhost
-port
8047
-username
admin
-password
######
-input_file_path 
d:\\mlcp_data\\
-output_uri_replace
"mlcp_data"

Ran the mlcp.bat file in command prompt but could not see any xml files(not even small files) being uploaded in database .
We are getting error : "Batch Recursion exceeds Stack limits" and "Batch processing is aborted".

Please help and suggest .

Are you XML files aggregates? That is, is your intention that a large (2GB) file will be turned into many smaller documents in the database? — Dave Cassel
Hi Dave , The issue is not only of large file , even files as small as 7.5MB are not getting uploaded using mlcp. We can fragment large files into small fragment and then load it . — Reecha Bhandari

grtjn grtjn · Accepted Answer · 2015-10-15T19:23:13

The error message in step 5 indicates that you have a batch file that is calling itself recursively. It sounds like you accidentally edited d:\mlcp-1.3-3-bin\mlcp-1.3-3\bin\mlcp.bat to make it call itself recursively.

You better choose a different batch file name to initiate MLCP with the correct options.

In addition to the comment from Dave, I really recommend looking at the MLCP support for splitting large XML files. MarkLogic works better with relatively small XML fragments, on the record or book or object level. You can find more details about that here:

http://docs.marklogic.com/guide/ingestion/content-pump#id_65814

HTH!

XML file upload using MLCP

1 Answers