.Net core HttpClient bug? SocketException: An existing connection was forcibly closed by the remote host

Question

The following code runs without any error in a full .Net framework console program. However, it got the following error when running in .Net core 2.1.

System.AggregateException
  HResult=0x80131500
  Message=One or more errors occurred.
  Source=System.Private.CoreLib
  StackTrace:
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at ConsoleApp1.Program.requestString(String url) in C:\source\repos\ConsoleApp1\Program.cs:line 38
   at ConsoleApp1.Program.Main(String[] args) in C:\source\repos\ConsoleApp1\Program.cs:line 13

Inner Exception 1:
HttpRequestException: The SSL connection could not be established, see inner exception.

Inner Exception 2:
IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.

Inner Exception 3:
SocketException: An existing connection was forcibly closed by the remote host

Code:

class Program
{
    static void Main(string[] args)
    {
        var url = "https://google.com";
        var (statusCode, html) = requestString(url);
        Console.WriteLine("%d %s", statusCode, html);
    }

    static CookieContainer cc = new CookieContainer();

    static HttpClientHandler handler = new HttpClientHandler { AllowAutoRedirect = false, CookieContainer = cc };

    public static async Task<(int statusCode, string content)> requestStringAsync(string url)
    {
        ServicePointManager.ServerCertificateValidationCallback += (sender, cert, chain, sslPolicyErrors) => true;
        ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls12;
                                                                     // | SecurityProtocolType.Ssl3;
        using (var request = new HttpRequestMessage { RequestUri = new Uri(url), Method = HttpMethod.Get })
        using (var client = new HttpClient(handler))
        {
            var response = await client.SendAsync(request); // Error (actual line)
            // response.EnsureSuccessStatusCode() |> ignore
            var statusCode = (int)response.StatusCode;
            var content = await response.Content.ReadAsStringAsync();
            return (statusCode, content);
        }
    }

    public static (int statusCode, string content) requestString(string url)
    {
        return requestStringAsync(url).Result;
    }
}

Answer 1

There is a bug for .NET Core 2.1 Preview mentioning this issue. That may be the cause. However, I also notice that your setting of TLS is incorrect. You're currently enabling it, but overwriting all other protocols that have been set. Instead of this:

ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls12;

You should be using this:

ServicePointManager.SecurityProtocol |= SecurityProtocolType.Tls | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls12;
// ----------------------------------^

I think that's a side issue, but worth fixing all the same.

Update

The referenced GitHub issue above has a discussion which eventually links to the official announcement for .NET Core 2.1 SDK Preview 2. It has the following to say:

Sockets Performance and SocketsHttpHandler

We made major improvements to sockets in .NET Core 2.1. Sockets are the basis of both outgoing and incoming networking communication. The higher-level networking APIs in .NET Core 2.1, including HttpClient and Kestrel, are now based on .NET sockets. In earlier versions, these higher-level APIs were based on native networking implementations.

...

You can use one of the following mechanisms to configure a process to use the older HttpClientHandler:

From code, use the AppContext class:

AppContext.SetSwitch("System.Net.Http.UseSocketsHttpHandler", false);

The AppContext switch can also be set by config file.

The same can be achieved via the environment variable DOTNET_SYSTEM_NET_HTTP_USESOCKETSHTTPHANDLER. To opt out, set the value to either false or 0.

Answer 2

In my case the root cause was using multiple concurrent calls of HttpClient.GetAsync(url). I fixed it by making HttpClient instance as singleton.

More info here.

Therefore, HttpClient is intended to be instantiated once and reused throughout the life of an application. Instantiating an HttpClient class for every request will exhaust the number of sockets available under heavy loads. That issue will result in SocketException errors. Possible approaches to solve that problem are based on the creation of the HttpClient object as singleton or static...