System.Net.Http.HttpRequestException Error while copying content to a stream

27
votes

I am using the HttpClient class in .NET Framework 4.5.2.

I calling PostAsync against a third party web service. 80% of the time this post works, 20% of the time our response is cut short. In this situation we get the following exception:

System.Net.Http.HttpRequestException: Error while copying content to a stream. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host at System.Net.Sockets.NetworkStream.BeginRead(Byte[] buffer, Int32 offset, Int32 size, AsyncCallback callback, Object state) --- End of inner exception stack trace --- at System.Net.Sockets.NetworkStream.BeginRead(Byte[] buffer, Int32 offset, Int32 size, AsyncCallback callback, Object state) at System.Net.FixedSizeReader.StartReading() at System.Net.Security._SslStream.StartFrameHeader(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security._SslStream.StartReading(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security._SslStream.ProcessRead(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.TlsStream.BeginRead(Byte[] buffer, Int32 offset, Int32 size, AsyncCallback asyncCallback, Object asyncState) at System.Net.ConnectStream.BeginReadWithoutValidation(Byte[] buffer, Int32 offset, Int32 size, AsyncCallback callback, Object state) at System.Net.ConnectStream.BeginRead(Byte[] buffer, Int32 offset, Int32 size, AsyncCallback callback, Object state) at System.Net.Http.HttpClientHandler.WebExceptionWrapperStream.BeginRead(Byte[] buffer, Int32 offset, Int32 count, AsyncCallback callback, Object state) at System.Net.Http.StreamToStreamCopy.StartRead()

A subsequent identical request succeeds.

We cannot retry this request as the business action has already been taken. So it leaves us in an awkward situation.

This is my code:

using (var httpClient = new HttpClient())
{
    httpClient.DefaultRequestHeaders.Authorization = authorizationHeader;
    HttpContent httpContent = new StringContent(someXml);

    //Exception occurs on next line...
    var response = await httpClient.PostAsync("https://thirdpartyendpoint", httpContent);
    var responseXml = await response.Content.ReadAsStringAsync();  
    //convert to Dto              
}

The third-party service are successfully saving the record to their database and do not see any obvious exceptions at their end. They did note that the failing requests generally took longer (around 18-30 seconds) to write to the database than the successful requests.

What can I do to handle this better?

3
c#asp.net.netasp.net-web-api
Seems there is a blocking operation or the bad indexes in their database. Try to profile database queries or monitor db locks. – Roman Dibikhin
Hi Roman, despite the additional delay, they do not raise an exception in their code and they complete the response. We just do not receive it. – jonho
They would check HTTP connection timeout on their side. – Roman Dibikhin

3 Answers

21
votes

we resolved this problem with 2 code changes:

  1. Dispose of the httpResponseMessage and just work with a simple DTO

    using (var httpResponseMessage = await httpClient.SendAsync(httpRequestMessage))
{
    return await CreateDto(httpResponseMessage);
}

  2. Downgrade the version of HTTP to v1.0

    var httpRequestMessage = new HttpRequestMessage(HttpMethod.Post, new Uri(url))
{
    Version = HttpVersion.Version10,
    Content = httpContent
};

await client.SendAsync(httpRequestMessage);

which has the effect of adding this Http header

Connection: close

rather than this

Connection: keep-alive
4
votes

I had a similar problem with the use of a shared HttpClient connecting to a server for REST calls. The problem ended up being a mismatch between the KeepAlive timeout on the client and server. The client side timeout is set by the MaxServicePointIdleTime setting on the ServicePointManager and defaults to 100s. The server side idle timeout was set to a shorter value in our server.

Having a shorter timeout on the server as compared to the client resulted in the server sporadically closing a connection just when the client was attempting to connect. This resulted in the reported exception.

Note that I ultimately found the problem because I also received this exception under the same conditions:

System.Net.WebException: The underlying connection was closed: A connection that was expected to be kept alive was closed by the server.
2
votes

I had the same error (Error while copying content to a stream) with HTTPClient PutAsync() method:

using (StreamContent content = new StreamContent(stream))
{
    HttpResponseMessage response = await client.PutAsync(url, content))
}

You need to specify the HttpCompletionOption.ResponseHeadersRead flag which is not available in PutAsync so I switched to SendAsync:

using (StreamContent content = new StreamContent(stream))
{
    var httpRequest = new HttpRequestMessage(HttpMethod.Put, url);
    httpRequest.Content = content;

    HttpResponseMessage response = await client.SendAsync(httpRequest, HttpCompletionOption.ResponseHeadersRead);
}