Efficient & Scalable connected TCP Windows Service using C# .Net 4.5

Question

Requirements: The need is for a windows service based C# .NET 4.5 always (at least long) connected TCP Server architecture with vertical and horizontal scaling and each server may handle max possible connections. Clients can be any IoT (internet of things).

I am aware of the limitations on ports but still wonder why these limitations in this era of tech (we always have limits but why still the old ones?!). Also temporary tcp/http connections will scale fine but not a requirement here.

Design:

• Single thread per server for async-accept new connections (lifetime of server). code: rawTcpClient = await tcpListener.AcceptTcpClientAsync();

• One thread per client connection (loop) to hold client connection ? (see my Q below)

• a Task for performing client operation (short term, intermittent
  operations)

my Question on optimization (if Possible?): How can I optimize/manage to hold all the client connections in a set of threads/threadpool instead of one thread per connection since this is client-lifetime which may be for a long duration?
Ex: per server, only 50 threads based tasks allocated to hold connected clients so that they don't get disconnected, while waiting for client data?

Answer 1

Efficient & Scalable

The very first thing you need to decide is how efficient you want to be. The socket APIs can get extremely complex if efficiency is your top priority. However, efficiency is almost never the top priority, even though a lot of people think it is. The problem is that complexity can increase exponentially with efficiency/scalability, and if you simply maximize efficiency/scalability, you'll end up with an almost unmaintainable system. So you'll need to decide where to draw the line on that scale.

Particularly if you have horizontal scaling, you probably don't need to use the extreme-efficiency socket APIs.

I am aware of the limitations on ports but still wonder why these limitations in this era of tech (we always have limits but why still the old ones?!).

Compatibility. Ports in particular are represented by a 16-bit value. The only way this would change is if a new standard came out, and everything upgraded. NICs, gateways, ISPs, and IoT devices. That's a high order, and will probably never happen.

Single thread per server for async-accept new connections (lifetime of server).

That's fine. If you have a large amount of connection turnover, you can have multiple accept threads, too. Just keep your backlog high (it should be high by default on Windows Server OSes).

One thread per client connection (loop) to hold client connection?

Er, no.

You'll want to use asynchronous I/O, for sure.

You should have continuous (asynchronous) reads going on all connected clients, and then do (asynchronous) writes as necessary. Also, if the protocol permits it, you should periodically write heartbeat messages to each connected client; otherwise, you'll need a timer for each client to drop the connection. Depending on the nature of your writes, you may need to have a queue of pending writes per client.

a Task for performing client operation (short term, intermittent operations)

If you use asynchronous tasks, then all your actual code will just run on whatever threadpool thread is available. No need for dedicated tasks at all.

You may find my TCP/IP .NET Sockets FAQ helpful.