invalid content length while importing 2.6 GByte xml in solr

Question

I am getting java.lang.IllegalArgumentException: invalid content length error while importing 2.6 GByte xml in solr . command i used is java -Durl=http://solrserver.local:8080/solr/solr1/update -jar post.jar dssolrprice.xml . the maxHttpHeaderSize is 10485760 in tomcat server.xml. what could be the reason

Answer 1

Use DataImportHandler instead, it works in a streaming fashion. You should not be sending files that big over an internet connection.

Answer 2

There are some limit when you post something via HTTP. i.e. I think that default size for HTTP post at Tomcat is 2 MB.

1) So maybe you can consider DataImportHandler for such kind of purposes.

2) On the other hand you can use a SolrJ client and split your data into chunks(if you can) and send them to index.