How do I read stdout/stderr output of a child process correctly?

Question

I have written a program a.exe which launches another program I wrote, b.exe, using the CreateProcess function. The caller creates two pipes and passes the writing ends of both pipes to the CreateProcess as the stdout/stderr handles to use for the child process. This is virtually the same as the Creating a Child Process with Redirected Input and Output sample on the MSDN does.

Since it doesn't seem to be able to use one synchronization call which waits for the process to exit or data on either stdout or stderr to be available (the WaitForMultipleObjects function doesn't work on pipes), the caller has two threads running which both perform (blocking) ReadFile calls on the reading ends of the stdout/stderr pipes; here's the exact code of the 'read thread procedure' which is used for stdout/stderr (I didn't write this code myself, I assume some colleague did):

DWORD __stdcall ReadDataProc( void *handle )
{
    char buf[ 1024 ];
    DWORD nread;
    while ( ReadFile( (HANDLE)handle, buf, sizeof( buf ), &nread, NULL ) &&
            GetLastError() != ERROR_BROKEN_PIPE ) {
        if ( nread > 0 ) {
            fwrite( buf, nread, 1, stdout );
        }
    }
    fflush( stdout );
    return 0;
}

a.exe then uses a simple WaitForSingleObject call to wait until b.exe terminates. Once that call returns, the two reading threads terminate (because the pipes are broken) and the reading ends of both pipes are closed using CloseHandle.

Now, the problem I hit is this: b.exe might (depending on user input) launch external processes which live longer than b.exe itself, daemon processes basically. What happens in that case is that the writing ends of the stdout/stderr pipes are inherited to that daemon process, so the pipe is never broken. This means that the WaitForSingleObject call in a.exe returns (because b.exe finished) but the CloseHandle call on either of the pipes blocks because both reading threads are still sitting in their (blocking!) ReadFile call.

How can I solve this without terminating both reading threads with brute force (TerminateThread) after b.exe returned? If possible, I'd like to avoid any solutions which involve polling of the pipes and/or the process, too.

UPDATE: Here's what I tried so far:

Not having b.exe inherit a.exe; this doesn't work. the MSDN specifically says that the handles passed to CreateProcess must be inheritable.
Clearing the inheritable flag on stdout/stderr inside b.exe: doesn't seem to have any effect (it would have surprised me if it did).
Having the ReadDataProc procedure (which reads on both pipes) consider whether b.exe is actually running in addition to checking for ERROR_BROKEN_PIPE. This didn't work of course (but I only realized afterwards) because the thread is blocked in the ReadFile call.

@adf88: according to the MSDN page on WaitForMultipleObjects, it can way on all kinds of things - but not pipes. My experiments seem confirm this: the read end of the pipe is always signalled, even when there is no data available. The CancelSynchronousIO() function looks nice, but it's not an option for me since it's only available on Windows Vista and newer. — Frerich Raabe
@adf88: The question you reference looks similiar, but it's not a duplicate of this question. I'm facing the same symptom, but I'm asking for solutions in a much wider solution space than in that other question. — Frerich Raabe
It should be sufficient for b.exe to pass FALSE to the bInheritHandles argument of CreateProcess(). — Harry Johnston

adf88 adf88 · Accepted Answer · 2010-07-03T07:03:39

Use named pipe and asynchronous ReadFile
or
Parse the output read from the pipe looking for the end (it may be too complicated in your case).

How do I read stdout/stderr output of a child process correctly?

4 Answers