Is it possible to have a set of thread pools that share threads from a large thread pool instead of creating new thread?

2
votes

Is it possible to have a set of thread pools that share threads from a large thread pool instead of creating new thread?

In our RESTful API application, a request may involve several parallel tasks. To improve the performance, we want to execute parallel tasks in a thread pool which has a fixed number (say 200) of threads. But we also want to limit the max number of threads that each request can use. So I am thinking if it is possible to create a sub thread pool with a maximum pool size for each request, which does not create thread by itself but tries to fetch a new thread from the global thread pool and put job into queue if there is no available threads.

Has anyone done something similar? Or is there any other better solution?

Thanks!!

2
javamultithreadingconcurrencythreadpoolexecutorservice
Does each request need its own thread pool? Couldn't you just have a single thread pool that all the requests pull threads from? - Pace
I am not sure if each request has to have its own thread pool, but the problem with a single thread pool is that one request could involve 300 parallel tasks, and then it will consume all threads in the single thread pool, then the next request has no thread to use. That's not fair for the second request. So actually what I am trying to achieve is to set a max number of threads that one request can use. - Wei Cai

2 Answers

2
votes

Instead of thread pools, think of executors. An executor consists of 2 things: a job queue and a thread pool. What you need is a light-weight executor for each request which has a job queue but has no threads. Instead, it passes jobs from its queue to the main executor (which owns all the threads). The trick is that the light-weight executor counts how many jobs it has submitted to the main executor, and stops passing jobs when the number exceeds the limit. Each job, before being passed to the main executor, is wrapped in an object of type Runnable which a) has the reference to the parent light-weight executor, b) executes the wrapped job and c) notifies the referenced executor when the job is finished so that the executor can pass another job to the main executor (or just decrease the job counter, if there is no jobs in the queue).

0
votes

You could create a thread pool for every task e.g. Executors.newFixedThreadPool(10) This will do what you ask for with the inefficiency of potentially creating threads that a particular task instance doesn't need.