Optimizing Linux Socket

2
votes

I'd like to ask some questions about optimizing linux socket. I try to make a multithreaded loadbalancer by using boost and simple linux socket. The loadbalancer works just as simple as these steps:

  1. A request comes in and tcp listener will accept a socket, just say it clientSocket and create a new thread
  2. When the thread start, it will create a back-end socket, just say it serverSocket to the back end server (service)
  3. After serverSocket established, I spawn a new thread to read from serverSocket and send the data/response to clientSocket
  4. And for the main thread, I call a function that will read from clientSocket and send to serverSocket
  5. When one of those two sockets become invalid, the worker will close both sockets and dies

I also use Waitset from ting library, which is using epoll, to make the recv method in blocking mode, so that it will wait until there's an event occured and then read the data from the socket.

The problem is when I tested the loadbalancer with AB, -n 10000 -c 100 -k, the result is very disappointing. I only got ~1600 tps. I tried to log request time taken for each request, but the result was good. Each round-trip got < 1000 microsecs/1 milisecs.

But when I log for incoming request intervals, next request processed about > 5000 microsecs/5 milisecs from current request received. Maybe anyone can suggest a better solution to optimize the socket operation here? Thank you.

2
c++multithreadingperformancesocketsboost
What does this have to do with boost? Did you mean boost performance? - Ralf
I spawn the thread with boost::thread. What I'm saying here is about how to improve linux tcp socket performance. - Xzorax
Spawning a new thread every time you receive a TCP connection is not a very efficient way to do things. You might want to take a look at the C10K problem article ( kegel.com/c10k.html ) for ideas on alternatives to that. (At the very least you could implement a thread pool so that in most cases an already-running thread could be tasked with the work, rather than spawning a new one every time) - Jeremy Friesner
I don't understand what you're doing. Are you really creating two threads per connection? That's ridiculous. The way these things are done is with a thread bound on each CPU just handling epoll events. - cdleonard

2 Answers

6
votes

You are making this overly complex. A thread per connection does not scale beyond trivial examples, read the C10K problem for more details.

I suggest reading about the Boost.Asio library for your load balancer. It uses epoll(4) on Linux systems for asynchronous event demultiplexing and will scale much better than a thread per connection.

2
votes

Well, the problem lies in that you are creating one thread per connection. That won't scale out well. So why don't you just create one thread that just monitors the incoming connection request and the in/out/hup events with epoll. The thread does not do other things to make it simple and efficient. When data is available, pass it to the thread workers, which do the work. You can join the event thread and the thread workers (thread pool created when init) by in/out queues.

Well, if this is not efficient enough when you have a lot of connections, you can balance the connections in multi-processes. Then the model becomes that you fork several child processes during initialization, and pass the server socket to each of them. When a connection request comes in, each of the child processes have the chance to accept. The real load balancing with multi-process.

According the model above, 20,000+ connections in one server are not a problem. Hope that helps you :)