Number of threads to limit CPU usage

3
votes

My application is a "thread-per-request" web server with a thread pool of M threads. All processing of a single request runs in the same thread.

Suppose I am running the application in a computer with N cores. I would like to configure M to limit the CPU usage: e.g. up to 50% of all CPUs.

If the processing were entirely CPU-bound then I would set M to N/2. However the processing does some IO.

I can run the application with different M and use top -H, ps -L, jstat, etc. to monitor it.

How would you suggest me estimate M ?

2
javamultithreadingperformancejvmcpu-usage
Whatever M you choose, if M is more than M / 2, the server will handle N requests in parallel, and thus potentially consume more than 50% of the CPUs, if they're available. If you want a limit, the OS should set it, not the JVM. - JB Nizet
% of cpu used is an statistic not a resurce waste. Why not limit the process priority instead? Threads are not only to optimize in processing time but to make programing easier. You may have the treads you want and don't interfere much with other processes. - aalku
Why should one want to limit CPU usage? When it's there, use it. When it's insufficient, scale. If you can't scale, your business model has serious problems. - Markus W Mahlberg
Install a hardware virtual machine (en.wikipedia.org/wiki/…), and set it to consume 50% CPU time. - Alexei Kaigorodov

2 Answers

2
votes

To have a CPU usage of 50% does not necessarily mean that the number of threads needs to be N_Cores / 2. When dealing with I/O the CPU wastes many cycles in waiting for the data to arrive from devices.

So you need a tool to measure real CPU usage and through experiments, you could increase the number of threads until the real CPU usage goes to 50%.

perf for Linux is such a tool. This question addresses the problem. Also be sure to collect statistics system wide: perf record -a.

You are interested in your CPU issuing and executing as many instructions / cycle as possible (IPC). Modern servers can execute up to 4 IPC for intense compute bound workloads. You want to go as close to that as possible to get good CPU utilization, and that means that you need to increase the thread count. Of course if there's many I/O that won't be possible due to many context switches that bring some penalties (cache flushing, kernel code, etc.)

So the final answer would be just increase the thread count until real CPU usage goes to 50%.

0
votes

It depends enterely on your particular software and hardware. Hardware is important since if a thread blocks writing to a slow disk it will take long to wake up again (and consume cpu again) but if the disk is really fast the blocking will only switch cpu context but the thread will be running again immediately.

I think you can only try and try with different parameters or the app may monitor it's CPU use itself and adjust the pool dynamically.