Ruby Thread Pool theory

Question

I'm running a Ruby thread pool (Thread.pool(10) for example), and I'm running into trouble with how I should actually be handling the logic behind distributing tasks.

I want to run ten tasks at a time, which would normally work - however, pool.process{...] doesn't block the main thread after 10 threads are in use - it keeps loading up tasks (so my while loop is loading an insane amount of tasks into the pool, which isn't correct... I actually need to check to see if any of the ten tasks return a 'false' before loading more). So, simply put, I want to run ten tasks at a time and only load new tasks onto the pool if there's less than ten tasks running.

Should I not be using a pool for this?

Answer 1

After playing around with this a lot, I ended up having to restructure my code to better fit the logic of what I was doing. There are interesting functions for Thread.pool, such as trim, but the descriptions online didn't quite make enough sense to me to use them.

In short: pool.process should only be run once you've already checked that you want that subtask run.