Hyper-V PLINQ Virtual Machine Parallel issue

Question

I have a PLINQ query as such...

batch
.AsParallel()
.WithExecutionMode(ParallelExecutionMode.ForceParallelism)
.WithCancellation(cancellationToken);
.Select(i => new { instruction = i, accountKey = new AccountKey(i.x, i.y, i.z) })
.GroupBy(x => x.accountKey)
.ForAll(grouping =>
{
    foreach (var instructionBatch in grouping.OrderBy(i => i.instruction.FileRow).Select(i => i.instruction))
    {
        // High CPU method.
    }
});

In a batch, there can be 10,000 records. These make calls to the High CPU method which in turn is making calls out to web-services and saving info to databases.

On my physical 64bit pc i7-4770 CPU @ 3.40 GHz 16.0GB ram. The service where this code is run from fires up around 32 threads and sits around 150,000 - 200,000 KB memory usage.

On a Hyper-V test environment which is a 64 bit virtual machine E5-2630 v3 @2.40GHz it generates over 200 threads and memory hits nearly the 2gb limit.

Is there any reason why it is firing up so many threads and why memory is not being released on the virtual machine?

Do I need to use WithDegreeOfParallelism. If this process might be called simultaneously with 4 different batches (1 x 1 record, 1 x 100 records, 1 x 1000 and 1 x 10,000 for example) does this mean that when if I specify a WithDegreeOfParallelism the 4 batches will each fire with that number of threads, even the batch of 1 record?

Thanks for any assistance.

usr usr · Accepted Answer · 2015-04-14T10:01:22

The TPL Parallel and PLINQ facilities are not good at handling IO. They tend to chose bad thread counts. The thread count employed by these methods is heuristically driven. I believe it is the thread-pool that contains this heuristic.

When IO is in play I highly recommend using WithDegreeOfParallelism. You can use Environment.ProcessorCount. If there is IO involved you probably want to slightly over-subscribe and add a constant amount of threads.

In PLINQ WithDegreeOfParallelism is an absolute amount. No more, no less. So yes, 4 concurrent queries result in 4 times the number of threads. I believe this problem does not happen with the built-in automatic thread count heuristic.

Consider using a fixed concurrency TaskScheduler for all concurrent queries.

Here's an experiment: Run that loop with a Thread.Sleep(1000000). You'll find tons of threads. Probably, one every 500ms. This is the thread-pools way of injecting threads when it thinks more are needed to avoid deadlocks and increase utilization. Totally inappropriate for IO.

Hyper-V PLINQ Virtual Machine Parallel issue

2 Answers