Thread Pool Implementation

Question

I'd like to make a simple thread pool (on a *nix system) to handle input on my asynchronous communication server. And I have some questions. At first I just want to have a static number of threads, arbitrarily say 6.

How should I send the command to the worker thread?

I was thinking of using simple socket communication, with something like this

main thread:: send [(recv) socket (send) otherSocket];

then have my threads simply block on the receive call, is this horribly inefficient?

Is there a library I should use? Are there better implementation methods?

(later I am going to make the threads dynamic, I just figured it would be easier to start with a set number)

Answer 1

First of all, you should realize that Windows has not just one, but two thread pool APIs built in. In case that wasn't enough, I/O completion ports are also basically thread pools in disguise. As such, there's probably not a lot of reason to write your own thread pool at all.

Unless you're planning on allowing your threads to run across separate machines, I would not use sockets for communication. The normal intent with threads is that they should keep overhead to a minimum; sockets don't fit very well with that. If you're crossing machine boundaries, a lot more overhead (network communication) becomes nearly unavoidable, and in that case, using sockets instead of some other network API doesn't make a lot of difference. Also note that for cross-machine work, you need to do quite a bit more in general, marshalling data from one to another, and so on.

Assuming you're sticking to a single machine, I'd use some manner of thread-safe queue instead of a socket. I posted one in a previous answer that you might find useful. That includes some sample code that creates a thread pool to service tasks in the queue.

Answer 2

Is there a library you should use? Well, most certainly, otherwise you won't be accomplishing any of what you want in standard C++. This brings into question what platform you want to use... are you using Windows, or some flavor of Linux? If you're using Windows, you can use Windows threads and Win Socks 2. If Linux you can try Posix threading, and as for networking I'm not exactly sure. However, you could just use Boost libraries to do all of this, and it should work across Windows and your Linux OS.

As far as sending a message, you could use a mutex for your message queue, and just simply have a receive message function that locks it and adds the message, which works for both networked incoming messages from the socket on whatever thread, and for other threads wanting to send a message to that thread. Also don't forget, when the thread is processing messages, you also want to lock the mutex; anytime you're accessing that message queue, you need it locked to prevent collision/data errors.

Answer 3

Since you are using sockets, you will have to use something like select or poll to wait on the socket activity. So for inter-thread communication (itc)something that uses a file descriptor will be very useful. This might be a pair or pipes or a unix domain socket.

You will then have a your main thread accept a new connect and use your itc mechanism to pass the new file descriptor to the worker thread. The worker thread will then add the new file descriptor to the select/poll loop.

If you are connecting and waiting on a backend server (like a database) you may decide block while waiting for the recv to complete although you should probably avoid this. You server will be more responsive if you use a state-machine and roll up your stack back to the select look (after adding the backend file descriptor to the select/poll loop). This will make it easier for your main thread to control your worker thread. If you are clever, your threads may be able to process more than one request at a time and with only a handful of thread you should have a very scalable server.

Try to avoid using semaphore or mutexes for itc and rather send all requests through your itc pipes or unix domain sockets. You will save yourself a lot of heart-ache with race conditions.