1
votes

I have 5 threads (on a multicore system) that simultaneously wait for:

  1. A mutex M to be acquired
  2. An event E to be signaled

I'm using WaitForMultipleObjectsEx(..., TRUE, INFINITE, TRUE) since the threads use APCs.
Note that the APCs do not use/touch/alter/observe the mutex or the event in any way.

I'm using Visual Studio along with Process Hacker to debug, and I'm witnessing a strange phenomenon:
Frequently, the thread that is supposed to run (call it thread A) is still waiting on the two objects.

I say "supposed to run" because:

  1. I have verified many times (including via Process Hacker) that the E is indeed in a signaled state
  2. No other thread has acquired ownership of M. In fact, 3 other threads are also waiting to acquire M.

Nevertheless, according to Process Hacker (which uses NtQueryMutant to retrieve the count), during such scenarios, M has an absurdly low "Count" value, such as -618.

Of course, I'm not forgetting to call ReleaseMutex (yes, I'm checking the return value too); indeed, once someone later acquires the mutex, the Count increases back up.

So this begs the question: if 3 of the 4 other threads are sleeping, what is the last thread (B) doing?

The answer is that B is executing an APC inside the WaitForMultipleObjectsEx call.
No matter how far I "step" through B in Visual Studio (even all the way back up RtlDispatchAPC), none of the other threads wake up.
It is only after that APC has finished executing that any thread wakes up!

I find this quite bizarre, WaitForMultipleObjectEx's documentation says:

When bWaitAll is TRUE, the function's wait operation is completed only when the states of all objects have been set to signaled. The function does not modify the states of the specified objects until the states of all objects have been set to signaled. For example, a mutex can be signaled, but the thread does not get ownership until the states of the other objects are also set to signaled. In the meantime, some other thread may get ownership of the mutex, thereby setting its state to nonsignaled.

Is the documentation wrong, or is this a mistake on my part?

Why does WaitForMultipleObjectsEx acquire the mutex during the execution of an APC, even though it knows it will never acquire the mutex anyway (because executing an APC implies it will return WAIT_IO_COMPLETE, never acquiring the mutex)?

1
The large negative count on the mutex implies that the same thread has acquired it multiple times, probably in some recursive fashion. Why might this happen? Do your APCs do alertable waits?arx
@arx: That's the funny thing -- my APCs don't do alertable waits. The maximum nesting of APC's is 1. (I've verified this both in the code and inside Process Hacker.)user541686

1 Answers

0
votes

I cannot comment so I use answer...

You don't show any code but shouldn't the APC trigger the WaitForMultipleObjectsEx return with WAIT_IO_COMPLETION code in the thread that is running??