ERROR_BROKEN_PIPE cannot read process output produced before process termination

Question

I've a simply process redirection routine in Win32. The problem here is that, if I put a Sleep between reads from the child process stdout, as soon as the process terminates while I sleep, I simply miss the last bytes from the pipe that outputs a ERROR_BROKEN_PIPE. It seems that , as soon as the child process terminates, it's pipes and associated handles are closed and anything pending discarded. The only solution seems to ReadFile from the pipe as fast as possible, but this is more than a problem for me due to the design of the software.

int _tmain(int argc, _TCHAR* argv[])
{
    BOOL bSuccess;
    WCHAR szCmdLine[MAX_PATH];
    char chBuf[BUFSIZE];
    DWORD dwRead;
    HANDLE g_hChildStd_OUT_Rd2 = NULL;
    HANDLE g_hChildStd_OUT_Wr2 = NULL;
    SECURITY_ATTRIBUTES saAttr2; 
    STARTUPINFO si2;
    PROCESS_INFORMATION pi2;

    ZeroMemory( &si2, sizeof(si2) );
    si2.cb = sizeof(si2);
    ZeroMemory( &pi2, sizeof(pi2) );
    //create pipe
    saAttr2.nLength = sizeof(SECURITY_ATTRIBUTES); 
    saAttr2.bInheritHandle = TRUE; 
    saAttr2.lpSecurityDescriptor = NULL; 
    assert(CreatePipe(&g_hChildStd_OUT_Rd2, &g_hChildStd_OUT_Wr2, &saAttr2, 0));
    //create child process
    bSuccess = FALSE;
    memset(szCmdLine, 0, MAX_PATH);
    wsprintf(szCmdLine, L"c:\\myprocess.exe");
    ZeroMemory( &pi2, sizeof(PROCESS_INFORMATION) );
    ZeroMemory( &si2, sizeof(STARTUPINFO) );
    si2.cb = sizeof(STARTUPINFO); 
    si2.hStdOutput = g_hChildStd_OUT_Wr2;
    si2.hStdError = g_hChildStd_OUT_Wr2; // also add the pipe as stderr!
    si2.dwFlags |= STARTF_USESTDHANDLES;
    assert(CreateProcess(NULL, szCmdLine, NULL, NULL, TRUE, 0, NULL, NULL, &si2, &pi2));
    //read from pipe
    CloseHandle(g_hChildStd_OUT_Wr2);
    memset(chBuf, 0, BUFSIZE);
    for (;;) 
    { 
        DWORD dwRead = 0;
        DWORD bytes = 0;
        if (!PeekNamedPipe(g_hChildStd_OUT_Rd2,NULL,NULL,NULL,&bytes,NULL)) {
            //printf("Peek named pipe failed!");
            break;
        }

        if (bytes != 0) {
            if (!ReadFile( g_hChildStd_OUT_Rd2, chBuf, BUFSIZE, &dwRead, NULL)) 
            {
                printf("EOF!!\n");
                break;

            }
            else {
                chBuf[dwRead] = 0;
                printf("%s", chBuf);
            }

        } else {
            Sleep(5000);
        }

    }
    while(1) {
    Sleep(1000);
    printf("Lopp!\n");
    }
    return 0;
}

Any hint ? Is there a way to keep the process on hold, like it happens in POSIX, until its pipes are read ?

Thanks!

You have to do it explicitly, call FlushFileBuffers() before you terminate. — Hans Passant
@HansPassant: that doesn't appear to be necessary. I can't reproduce the problem, even by having the child process terminate itself rather than exiting cleanly. Unless the child is actually calling DisconnectNamedPipe or something equally bizarre, I can't imagine what it could be doing to cause this. — Harry Johnston
@Leonardo: assuming the problem is indeed with the child process and can't be resolved, the best workaround is probably to not close g_hChildStd_OUT_Wr2. The catch is that this means ReadFile/PeekNamedPipe won't tell you when the child process has exited, so you'll have to check that separately. That means using polling, which is inefficient (or asynchronous I/O, which is complicated) but if you can't find another solution it may be the only way forward. — Harry Johnston
I'm considering moving the read operation on another blocking thread, that way I can read as fast as possible.... — Leonardo Bernardini
That would still be a race condition, because the thread might not get scheduled in time. It might work most of the time but wouldn't be reliable. — Harry Johnston

Serge Ballesta Serge Ballesta · Accepted Answer · 2015-01-27T13:40:40

I think the problem is not there.

You do have some problems in your code. Many are minors : you initialize twice si2 and pi2 (harmless), you have an endless loop so your prog will never end (I assume you Ctrl-C ...), you are mixing _tmain and WCHAR (should be either wmain and WCHAR or _tmain and TCHAR) but if you do an unicode build it is fine.

One is more serious : chBuf has size BUFSIZE, you read upto BUFSIZE chars in it (fine till here). But if you do read BUFSIZE chars, dwRead is BUFSIZE and on next line you have a buffer overflow

chBuf[dwRead] = 0; // buffer overflow if BUFSIZE chars have been read !

I only fixed that with :

if (!ReadFile( g_hChildStd_OUT_Rd2, chBuf, BUFSIZE - 1, &dwRead, NULL))

and the program works correctly.

BTW : the child process is knows by the system (even if terminated) until all handles to it have been closed ... and you have a handle to it in pi2.hProcess.

Of course I could not use c:\\myprocess.exe and tested with cmd /c dir. So if the above fix is not enough try to use same child as I did because the problem could come from myprocess.exe that is started with a NULL hStdInput

ERROR_BROKEN_PIPE cannot read process output produced before process termination

2 Answers