WSO2 ESB: retry HTTP request after timeout

Question

I'm using WSO2 ESB to route API requests send to an application hosted on Heroku, through proxy service.

In the last few days, we've noticed connections to the Heroku app would timeout from time to time, resulting in the failure of the whole client process.

Those requests are part of a long process (video transcoding), so relaunching the entire process just because 1 out of 40 requests failed is not really an option. It would be best to retry every failed request automatically a second (or n) time(s).

Implementing "retry on timeout" directly into client code is a mess, so we thought the could handle it for us by configuring the endpoint to retry all timed out requests. The WSO2 ESB documentation on endpoint error handling, advertises it's possible by setuping a failover group that contains only one leaf endpoint, dealing with my exact use case when "even rare message failures are not acceptable".

However, the provided sample configuration seem to continue dropping messages:

<endpoint name="SampleFailover">
    <failover>
        <endpoint name="Sample_First" statistics="enable" >
            <address uri="http://localhost/myendpoint" statistics="enable" trace="disable">
                <timeout>
                    <duration>60000</duration>
                </timeout>

                <markForSuspension>
                    <errorCodes>101504, 101505, 101500</errorCodes>
                    <retriesBeforeSuspension>3</retriesBeforeSuspension>
                    <retryDelay>1</retryDelay>
                </markForSuspension>

                <suspendOnFailure>
                    <initialDuration>1000</initialDuration>
                    <progressionFactor>2</progressionFactor>
                    <maximumDuration>64000</maximumDuration>
                </suspendOnFailure>

            </address>
        </endpoint>
    </failover>
</endpoint>

I'm still seeing log entries in wso2carbon.log such as:

[2016-04-13 16:02:50,576]  WARN -  Expiring message ID : urn:uuid:fd515b80-8d67-47e5-b409-0ec39ed58fc6; dropping message after timeout of : 60 seconds {org.apache.synapse.core.axis2.TimeoutHandler}

What am I missing here ?

Hi Erwan, did you figured out how to solve this, and what how to perform an actual retry at the endpoint level? After reading several times the documentation the retry parameter is actually just the number of failed called after which the endpoint will be suspended, not a retry of the message sending. — Nicolas

Nicolas Nicolas · Accepted Answer · 2017-05-24T10:02:54

I add some difficulties with this topic, and it is yet not clear if the endpoint can itself retrigger and timed out call.

After reading this tutorial : https://www.yenlo.com/blog/wso2torial-error-handling-in-wso2-esb-with-wiremock?success=true, it seems that yes, but I'm not sure...

Have you already tried to remveo the error codes ?

<timeout>
    <duration>60000</duration>
    <responseAction>fault</responseAction>
</timeout>
<suspendOnFailure>
    <initialDuration>1000</initialDuration>
    <progressionFactor>2.0</progressionFactor>
</suspendOnFailure>
<markForSuspension>
    <retriesBeforeSuspension>3</retriesBeforeSuspension>
    <retryDelay>1000</retryDelay>
</markForSuspension>

WSO2 ESB: retry HTTP request after timeout

1 Answers