I am using Orbeon pipelines as a middle tier in my application (running in Tomcat v6), which involves posting XML content to a MarkLogic database on the back-end. This is generally working well apart from cases where a substantial amount of content (5+MBs) is being posted to MarkLogic, which then processes this content before responding with the amended result document; this is a process that can take up to 10+ minutes.
I'm finding that when I run this process with large XML files with my local installation of Orbeon/Tomcat when communicating with an external MarkLogic server it works ok. However, when I run it on a server-based installation of Orbeon/Tomcat communicating with the same MarkLogic server, I get a "java.net.SocketException: Connection reset" error in the log before the MarkLogic server has had a chance to respond to Orbeon with the updated XML content. The server-based installation has (as far as I can tell) the same set-up as my local installation.
This reset seems to happen sometime after the 5-minute mark of when the initial submission to MarkLogic was made and the connection opened.
This isn't a problem with smaller files that require less time for MarkLogic to process and respond to Orbeon.
The Orbeon pipeline code that performs this submission to MarkLogic looks like the following:
<p:processor name="oxf:xslt">
<p:input name="data" href="#document" />
<p:input name="config">
<xforms:submission xsl:version="2.0" method="post" xxforms:username="$USERNAME" xxforms:password="$PASSWORD">
<xsl:variable name="parameters" as="xs:string*">
<xsl:value-of select="concat('modifiedTime=', doc('input:documentDate')/an:element)" />
<xsl:value-of select="concat('generate-effect-id=',
if (doc('input:review') = 'review') then 'true'
else (
if (doc('input:documentDate')/an:element[@an:attribute = 'some value']) then 'false'
else 'true'
)
)" />
</xsl:variable>
<xsl:attribute name="action" select="concat('http://db:$DBPORT/queries/some-query.xq?', string-join($parameters, '&'))" />
</xforms:submission>
</p:input>
<p:input name="review" href="#instance#xpointer(/parameters/review)" />
<p:input name="documentDate" href="#documentXML" />
<p:output name="data" id="submission" />
</p:processor>
<p:processor name="oxf:xforms-submission">
<p:input name="submission" href="#submission" />
<p:input name="request" href="#inputDoc" />
<p:output name="response" id="outputDoc" />
</p:processor>
And the Java stack error message that I get in the Orbeon log begins as follows:
ERROR XFormsServer - xforms-submit-error - setting throwable {throwable: "java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:168)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
at org.apache.commons.httpclient.HttpParser.readRawLine(HttpParser.java:78)
at org.apache.commons.httpclient.HttpParser.readLine(HttpParser.java:106)
at org.apache.commons.httpclient.HttpConnection.readLine(HttpConnection.java:1116)
at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.readLine(MultiThreadedHttpConnectionManager.java:1413) [...]
I have set the default timeouts for MarkLogic and Apache (which is the server being used) to 30 mins so I'm pretty sure this isn't the issue (and these match on both the local and server set-ups) and no timeout has been set for Orbeon, which according to the documentation (https://doc.orbeon.com/configuration/properties/general.html#socket-timeout) means there isn't one.
Any suggestions on where the problem may lay would be much appreciated, hopefully, there's something obvious that I've missed.
