How to choose thread pool size?

1
votes

Suppose I have to read, process and update a lot of files in Java. I am going to use one computer with 16 cores. Since I have both IO-bound (read and update files) and CPU-bound (processing) tasks I allocate 2 thread pools.

I would allocate one pool for CPU-bound tasks with 16 threads (the number of threads == the number of CPUs). Now I wonder what the IO-bound pool size is. Thread pools of what sizes would you suggest ?

1
javamultithreadingthreadpool
I would suggest testing. Can't beat actual results you know. – Kayaman
I'd say the number of processing cores available... – Thihara
I would suggest to leave at least 1 CPU for the operating system, then do testing with available tools to check where the bottle-neck in an real-world aproximation – RamonBoza
@Thihara Thanks. I will update the question. – Michael
I never would count on Java about management of threads, for low-level management, you may need to switch to something more reliable, like pthread. – user2511414

1 Answers

4
votes

It would depend on your storage capabilities and what kinds of IO you are trying to do. For example, long sequential writes on harddisks would favor a single IO thread, but you would want to scale this according to your requirements.

In this answer, https://stackoverflow.com/a/2821025/2855891, BlackAura explains why experimentation and profiling is probably the only way you can really find out.

There are probably already very good articles on this topic.