Netty server boss and worker thread pool sizes

Question

If my understanding with Netty server is correct, the main boss event loop thread pool (default size is 2*availableProcessors) accepts client connections and then offloads request processing work to worker threads.

Now, the question I have is, what should be the thread pool size for worker threads? If worker is doing some blocking operating say waiting for a network call response, then shouldn't the worker thread pool be large enough (say, 200 threads) to handle concurrent clients requests since each worker thread is blocked serving a client request? I see most of the ServerBootstrap examples showing working thread pool size to be 2xAvailableProcessors, which is same as main thread pool size. Won't this limit number of concurrent client request processing to 2xAvailableProcessors?

Adding to my confusing, I see one more thread pool being used called handler thread pool (EventExecutorGroup) and says that this thread pool handles request processing. Then whats the use of worker thread pool?

The three types of thread pools are created as below in the examples I came across.

EventLoopGroup bossGroup = new NioEventLoopGroup(1);
EventLoopGroup workerGroup = new NioEventLoopGroup(10);
EventExecutorGroup handlerThread = new DefaultEventExecutorGroup(20);

And if worker threads are doing some CPU intensive process (as opposed to waiting for network call response in above example), worker threads will contend with main boss thread. Won't this scenario make boss thread slow thereby client connections waiting in queue to be accepted or rejected? In this scenario, the server going to perform same as traditional blocking servers, I believe. Could you share your thoughts on this?

spinlok spinlok · Accepted Answer · 2020-10-20T05:05:56

Now, the question I have is, what should be the thread pool size for worker threads? If worker is doing some blocking operating say waiting for a network call response

You should never block on netty worker threads, especially for network I/O -- that is the entire point of using a non-blocking asynchronous framework like netty. That said, in order to interop with synchronous/blocking code like JDBC, netty provides a way to offload such execution into a separate threadpool using an EventExecutorGroup. Have a look at this snippet from Netty docs:

 // Tell the pipeline to run MyBusinessLogicHandler's event handler methods
 // in a different thread than an I/O thread so that the I/O thread is not blocked by
 // a time-consuming task.
 // If your business logic is fully asynchronous or finished very quickly, you don't
 // need to specify a group.
 pipeline.addLast(group, "handler", new MyBusinessLogicHandler());

And if worker threads are doing some CPU intensive process

This is a more general question, not exactly related to netty, but if your workload is heavily CPU intensive, using a async network I/O framework may not be a good fit for you.

Netty server boss and worker thread pool sizes

1 Answers