How to force GetQueuedCompletionStatus() to return immediately?

2
votes

I have hand-made thread pool. Threads read from completion port and do some other stuff. One particular thread has to be ended. How to interrupt it's waiting if it hangs on GetQueuedCompletionStatus() or GetQueuedCompletionStatusEx()?

  • Finite timeout (100-1000 ms) and exiting variable are far from elegant, cause delays and left as last resort.
  • CancelIo(completionPortHandle) within APC in target thread causes ERROR_INVALID_HANDLE.
  • CancelSynchronousIo(completionPortHandle) causes ERROR_NOT_FOUND.
  • PostQueuedCompletionStatus() with termination packet doesn't allow to choose thread.
  • Rough TerminateThread() with mutex should work. (I haven't tested it.) But is it ideologically good?
  • I tried to wait on special event and completion port. WaitForMultipleObjects() returned immediately as if completion port was signalled. GetQueuedCompletionStatus() shows didn't return anything.

I read Overlapped I/O: How to wake a thread on a completion port event or a normal event? and googled a lot.

Probably, the problem itself – ending thread's work – is sign of bad design and all my threads should be equal and compounded into normal thread pool. In this case, PostQueuedCompletionStatus() approach should work. (Although I have doubts that this approach is beautiful and laconic especially if threads use GetQueuedCompletionStatusEx() to get multiple packets at once.)

2
winapiio-completion-ports
"Rough TerminateThread": At the point you terminate you can't really know that the thread is suspended. Therefore it's unsafe. - usr
Why do you need to choose a particular thread? - usr
@usr indeed, even I wrap packet processing code and termination call in mutex, packet that is just taken from queue but not started to be processed may be lost. - George Sovetov
@Ben "just don't" is what I mentioned by saying "ideologically". Both of your suggestions are mentioned in question. - George Sovetov
You don't want to cancel IO, you just want the thread to exit, right? Best is to set a flag within the APC. When the GetQueuedCompletionStatusEx returns, you have to check the return code (because if there is an APC there may not be any work) then check the flag to see if you must exit. - Ben

2 Answers

3
votes

If you just want to reduce the size of the thread pool it doesn't matter which thread exits.

However if for some reason you need to signal to an particular thread that it needs to exit, rather than allowing any thread to exit, you can use this method.

If you use GetQueuedCompletionStatusEx you can do an alertable wait, by passing TRUE for fAlertable. You can then use QueueUserAPC to queue an APC to the thread you want to quit.

If the thread is busy then you will still have to wait for the current work item to be completed.

Certainly don't call TerminateThread.

1
votes

Unfortunately, I/O completion port handles are always in a signaled state and as such cannot really be used in WaitFor* functions.

GetQueuedCompletionStatus[Ex] is the only way to block on the completion port. With an empty queue, the function will return only if the thread becomes alerted. As mentioned by @Ben, the QueueUserAPC will make the the thread alerted and cause GetQueuedCompletionStatus to return.

However, QueueUserAPC allocates memory and thus can fail in low-memory conditions or when memory quotas are in effect. The same holds for PostQueuedCompletionStatus. As such, using any of these functions on an exit path is not a good idea.

Unfortunately, the only robust way seems to be calling the undocumented NtAlertThread exported by ntdll.dll.

extern "C" NTSTATUS __stdcall NtAlertThread(HANDLE hThread);

Link with ntdll.lib. This function will put the target thread into an alerted state without queuing anything.