How to execute a pool of tasks using set number of Java / Groovy threads without blocking

2
votes

Right now I have this Groovy code to run a series of tasks: 

    CountDownLatch latch = new CountDownLatch(tasks.size);

    for( task in tasks ) {
        Thread.start worker.curry(task, latch)
    }

    latch.await(300L, TimeUnit.SECONDS);

I'd like to limit the number of simultaneous threads to a certain number t. The way it's written now, for n tasks, n threads get created "at once". I thought about using multiple latches or some sort of callback, but couldn't come up with a good solution.

The solution should start new task threads as soon as running threads drops below t, until number running reaches t or there are no un-run tasks.

2
javamultithreadinggroovy

2 Answers

5
votes

You should check out GPars and use one of the abstractions listed. You can then specify the number to create in withPool(). I like fork/join:

withPool(4) { pool ->
  runForkJoin(rootTask) { task ->
    task.eachTask { forkChild(task) }
  }
}
4
votes

You can use the Executor framework. 

Executors.newFixedThreadPool(t);

This will create n and only n threads on start. Then you can submit to the executor to utilize these threads.

Edit: Thanks for the comment Josh, I'll post your solution

ExecutorService pool = Executors.newFixedThreadPool(6); 
for( task in tasks ) { 
    pool.execute Worker.curry(task)
} 
pool.shutdown();