Handle a large number of TCP Client Connections in .net

Question

I need to build an application (a server) that handles data sent from a Client via TCP. I must be able to support (at least) 2000 connections. I've made an attempt to write the TCP Server, but I find when I start to reach 600/700 connections, that the response from my server slows greatly (it actually slows down over time as more and more connections are received). I don't normally write networking code so I'm sure there are many concepts I've not fully comprehended and could be improved upon.

The main purpose of my server is to:

Handle data sent from clients and store it in a sql database.
Decide (based upon the last message received) what the correct response should be to the client.
Queue up a list of responses and send them to the client one at a time.

This needs to happen for all clients. Below is the code I have implemented:

    private readonly TcpListener tcpListener;
    private readonly Thread listenThread;    
    private bool run = true;

    public Server()
    {
         this.tcpListener = new TcpListener(IPAddress.Any, AppSettings.ListeningPort); //8880 is default for me.
         this.listenThread = new Thread(new ThreadStart(ListenForClients));
         this.listenThread.Start();
    }

    private void ListenForClients()
    {
         this.tcpListener.Start();

         while (run) {
              TcpClient client = this.tcpListener.AcceptTcpClient();

              //create a thread to handle communication with connected client
              Thread clientThread = new Thread(new ParameterizedThreadStart(HandleClientComm));
              clientThread.Start(client);
          }
    }
    private void HandleClientComm(object client)
    {
        Queue responseMessages = new Queue();
        while (run)
        {
            try
            {
                lastMessage = clientStream.GetMessage();

                if (lastMessage.Length > 0)
                {
                    // Process logic here...  
                    //an item may be added to the response queue, depending on logic.
                }

                if (responseMessages.Count > 0)
                {
                   clientStream.WriteMessage(msg);
                   clientStream.Flush();

                   // sleep for 250 milliseconds (or whats in the config).
                   Thread.Sleep(AppSettings.MessageSendDelayMilliseconds); 
                }
            }
            catch (Exception ex)
            {
                break;
            }
        }
        tcpClient.Close();
    }

And finally, here's an extension class I wrote to help me out:

namespace System.Net.Sockets
{
    using System.Text;

    public static class NetworkSteamExtension
    {
        private static readonly ASCIIEncoding Encoder = new ASCIIEncoding();

        public static string GetMessage(this NetworkStream clientStream)
        {
            string message = string.Empty;
            try
            {
                byte[] bMessage = new byte[4068];
                int bytesRead = 0;
                while (clientStream.DataAvailable)
                {
                    bytesRead = clientStream.Read(bMessage, 0, 4068);
                    message += Encoder.GetString(bMessage, 0, bytesRead);
                }
            }
            catch (Exception)
            {

            }
            return message;         
        }

        public static void WriteMessage(this NetworkStream clientStream, string message)
        {
            byte[] buffer = Encoder.GetBytes(message);
            clientStream.Write(buffer, 0, buffer.Length);
            clientStream.Flush();
        }
    }
}

Lots of articles on the subject people are using sockets instead. I've also read that .net 4.5 async / await is the best way to implement a solution.

I would like to make sure I take advantage of the newest features in .net (even 4.5.2 if it will help) and build a server that can handle at least 2000 connections. Can someone advise?

Many thanks!

Looks like you start a thread for every client connect. That will be 600/700 threads in your process that all need to get a timeslice.... check the Async methods of everything you want to do. — rene
So do you think all I need is a one line change? From "TcpClient client = this.tcpListener.AcceptTcpClient();" to "Task<TcpClient> client = this.tcpListener.AcceptTcpClientAsync();"? Or is there more to it? — Rob McCabe
Sounds like a potential use of Akka.Net. This framework makes it easy to scale and deals with all the complexities of async and threads. — Crowcoder

usr usr · Accepted Answer · 2015-08-09T12:29:43

OK, we need to fix some API usage errors and then the main problem of creating too many threads. It is well established that many connections can only be handled efficiently with async IO. Hundreds of connections counts as "too many".

Your client processing loop must be async. Rewrite HandleClientComm so that it uses async socket IO. This is easy with await. You need to search the web for ".net socket await".

You can continue to use synchronous accept calls. No downside there. Keep the simple synchronous code. Only make async those calls that have a significant avgWaitTime * countPerSecond product. That will be all socket calls, typically.

You are assuming that DataAvailable returns you the number of bytes in any given message. TCP does not preserve message boundaries. You need to do that youself. The DataAvailable value is almost meaningless because it can underestimate the true data that will arrive in the future.

It's usually better to use a higher level serialization protocol. For example, protobuf with length prefix. Don't use ASCII. You probably have done that only because you didn't know how to do it with a "real" encoding.

Dispose all resources using using. Don't use the non-generic Queue class. Don't flush streams, this does nothing with sockets.

Why are you sleeping?

Handle a large number of TCP Client Connections in .net

1 Answers