How to regard Hyper-Threading when deciding thread pool size?

7
votes

I've read several Questions and articles regarding how to decide on a size for a thread pool. Questions include:

Articles include:

However, none of these directly address the issue of Hyper-Threading on Intel chips.

On a computer with Hyper-Threading enabled, should I consider the virtual cores when deciding thread pool size?

Take for example Brian Goetz’ advice in the Java Concurrency In Practice book that, generally speaking, for a CPU-bound app one might consider using ( # of cores + 1 ) as the number of threads. For an Intel Core i7 chip with 4 real cores and 8 virtual (Hyper-Threading) cores, would that formula be ( 4 + 1 ) or ( 8 + 1 )?

Also, how much of difference does the nature of the app matter in how to consider the Hyper-Threading cores?

Contrary to the mention above, my own app is not CPU bound. Instead my app is a server-side Vaadin app where the threads are making Internet connections and hitting a local database through JDBC, several times a minute. Given that Hyper-Threading is basically a second set of registers attached to the same core, perhaps a CPU-bound app should consider only real cores while a network/IO-bound app should consider virtual cores?

Lastly, does the type of Intel chip affect Hyper-Threading and therefore the calculation of thread pool size? Specifically, is there a difference regarding this issue between a Xeon and a Core i7/i5? For example, the difference between a current MacBook (Core i7) and Mac Pro (Xeon).

I realize there are many variables involved, and this is a complex topic. No perfectly precise answers are possible. I'm just looking for some general rules and advice to help out programmers like me who are not as savvy about such hardware matters.

1
javamultithreadingthreadpoolhyperthreading
A network-bound APP doesn't benefit from more threads and should only choose the number needed to deal with things like unexpected page faults or how many I/O is can pend usefully at once. The CPU is almost irrelevant to an app that's not CPU bound. – David Schwartz
@DavidSchwartz, that's a rather broad claim, i'd say that in general, when you're exposed to external latencies (i.e. memory, network, etc), the more threads you have the more work you can achieve, assuming your app can parallelize its work. HT could help here by increasing the effective core count without the need of context switching. Of course, you also have caching considerations, this paper discusses the tradeoffs - ieeexplore.ieee.org/xpl/… – Leeor
For performance issues, the first step is always to PROFILE! – Nicu Stiurca
Your app didn't need nor worth to pay effort to consider whether true core or hyper-threaded core will benefit or not while it's 1/ not CPU bounded; 2/ abstracted by the Java VM implementation; 3/ Depends on the efficiency of the framework; 4/ More likely the database will face the bottleneck before the Java code running inside the framework. – Ken Cheung

1 Answers

3
votes

How to regard Hyper-Threading when deciding thread pool size?

The short answer is Don't.

The longer answer is that Goetz's "formula" is actually just a rule of thumb. The language used here

... "generally speaking, for a CPU-bound app one might consider using ( # of cores + 1 ) as the number of threads"

makes that clear. There are all sorts of reasons why that "rule of thumb" number might give you suboptimal performance.

The correct approach is:

  1. Pick a number
  2. Measure performance
  3. Adjust number and go to step 2.

.... until you arrive at a thread pool size that gives roughly the best answer for your use-case.

The other thing to note is that when you are building a server-based system is that performance is only one of many considerations. Another is how your system performs under extreme load. If you optimize your performance based on a "best case" workload without considering behaviour under overload situations, then you can be in for a nasty shock if something goes wrong.

Optimizing solely for maximum throughput can have bad consequences...