C# Cant reuse Http connections. No KeepAlive header. HttpWebRequest.KeepAlive = true;

Question

Ok. Here is the full code. I tried to reuse connections setting KeepAlive but it simply doesnt work. I looked in the Http messages using Feedler and Charles but all I can see is Connection: close in response.

I see 600 TCP connections in wait state opened by 10 threads. Each thread run one http requst at a time.

There is also bunch of responses which say - the unauthenticated request. The service requires digest authentication. The code obviously is static and simply run the same request few hundred times from different threads... So why some requestes fail to be authenticated?? I

    static void GetRest(string rest)
    {
        int i = Interlocked.Increment(ref counter);

        Uri uri = new Uri(rest);
        CredentialCache cc = new CredentialCache();
        cc.Add(uri, "Digest", new NetworkCredential("zz", "zz"));

        ServicePointManager.FindServicePoint(uri).SetTcpKeepAlive(true, 6000000, 100000);
        ServicePointManager.FindServicePoint(uri).ConnectionLimit = 5;

        while (!stop)
        {

            HttpWebRequest req = WebRequest.Create(rest) as HttpWebRequest;

            req.Credentials = cc;
            req.Method = "GET";
            req.Timeout = timeout;
            req.KeepAlive = true;

            try
            {
                using (HttpWebResponse res = (HttpWebResponse)req.GetResponse())
                {
                    StreamReader sr = new StreamReader(res.GetResponseStream());

                    string result = sr.ReadToEnd().Substring(0, 20);
                    int rc = Interlocked.Increment(ref responseCounter);
                    Console.Write(".");

                    Thread.Sleep(20);
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine("EXCEPTION {0}, {1}", i, ex.Message);
                Interlocked.Increment(ref badResponseCounter);
            }
        }

        Interlocked.Decrement(ref counter);
    }

I can't give much insight into your particular problem, but for this kind of thing I'd definitely suggest paring it down to a "minimum working example," then building up from there. For example: first try 1 authenticated HTTP request to this URI. Leave out the ServicePointManager stuff for now. Then try 600 of them in series (not parallel). Maybe at this point the ServicePointManager stuff needs to come back. Then try 10 in parallel. And so on. — Domenic
This question also seems relevant to your own: stackoverflow.com/questions/2179626/reuse-a-httpwebrequest — Domenic
Domenic. Thanks but the suggestion in the other thread say - close the Httpresponse which I do - I have using on it... The ServicePoint thing is necessary. Without it the whole thing works x3 slower — Boppity Bop

Darrel Miller Darrel Miller · Accepted Answer · 2010-07-06T01:48:38

If you are seeing Connection: Close in the response, does this not indicate to you that the server is forcing the connection to close after every request. I'm not sure there is anything you can do at this point to change that behaviour. Maybe you could ask the service provider why they are returning Connection: Close in every response.

C# Cant reuse Http connections. No KeepAlive header. HttpWebRequest.KeepAlive = true;

2 Answers