0
votes

EDIT: Just found out this is only an issue from a PowerShell console. It seems to work from cmd.exe. Investigating that more...

In a Windows GUI (/SUBSYSTEM:WINDOWS) app, it is relatively easy to attach or create a console, and just WriteFile to stdout. (This can get involved if wanting to use the standard C/C++ APIs, e.g. How do I get console output in C++ with a Windows program?, but if using the Win32 API WriteFile it seems simple enough).

However I notice when stdout is redirected, the code works fine in a console app, blows up if it's a GUI app.

Basic code is below, (or see full sample at https://github.com/billti/WinCons). Just include the below as a header file, and shortly after WinMain instantiate the class, call CreateConsole, then any call to Write will fail with the error marked with *** – but ONLY if launched with output redirected (e.g. myapp.exe > .\log.txt). It works fine if not redirected, and either way works fine in a console app.

Note: Attaching to the parent console can result in confusing interleaved output, but it does work.

Any idea how to resolve this? How can I get redirected stdout from a non-console Windows app to work?

#include <string>
 
class Log
{
public:
    void CreateConsole() {
        // In a console app, redirected or not, this returns a valid handle.
        // In a non-redirected GUI app, this returns NULL.
        // If redirected GUI (e.g. "winapp.exe > .\log.txt") this returns a valid handle.
        hStdOut = GetStdHandle(STD_OUTPUT_HANDLE);
        if (hStdOut == NULL || hStdOut == INVALID_HANDLE_VALUE) {
            // If launched from a console, then Attach work, else must Alloc.
            if (!AttachConsole(ATTACH_PARENT_PROCESS)) {
                if (!AllocConsole()) {
                    throw std::exception("Failed to allocate a console");
                }
            }
            hStdOut = GetStdHandle(STD_OUTPUT_HANDLE);
            if (hStdOut == INVALID_HANDLE_VALUE || hStdOut == NULL) {
                throw std::exception("Invalid stdout");
            }
        }
        hasConsole = true;
    }
 
    void Write(const std::string& msg) {
        DWORD written = 0;
        DWORD err = 0;
        if (hasConsole) {
            if (!WriteFile(hStdOut, msg.c_str(), msg.length(), &written, nullptr)) {
                // *** If output is being redirected in a GUI app, this always fails with ***
                //   0xE8 ERROR_NO_DATA "The pipe is being closed."
                err = GetLastError();
                throw std::exception("Failed to write to console");
            }
        }
    }
 
private:
    bool hasConsole = false;
    HANDLE hStdOut = INVALID_HANDLE_VALUE;
};
look like the powershell.exe exit. close self end of pipe. as result you and get STATUS_PIPE_CLOSINGRbMm
In PowerShell, myapp.exe > .\log.txt sets the child stdout to a pipe instead of a direct handle for the file. For a console application, PowerShell reads from its end of the pipe, converts the text encoding to the file encoding (including LF -> CRLF translation), and writes to the "log.txt" file. For a GUI process, it won't wait and read from the pipe, and it immediately closes its end of the pipe. The GUI child still has its stdout set to an open pipe via STARTUPINFO w/ STARTF_USESTDHANDLES, which AttachConsole and AllocConsole wouldn't replace even if you called them.Eryk Sun
CMD implements I/O redirection by modifying its own standard handles temporarily before calling CreateProcessW. It does not use the STARTUPINFO standard handles w/ STARTF_USESTDHANDLES. It also does not set itself up as a middle man for redirected I/O, so when redirecting stdout to "log.txt", the child directly inherits a valid handle for the file. OTOH, because CMD doesn't use the STARTUPINFO standard handles, calling AttachConsole or AllocConsole in the child will replace the stdout handle.Eryk Sun
So.... not (easily) solvable? (If interested, I opened an issue at github.com/PowerShell/PowerShell/issues/13706 also)Bill Ticehurst
But then it would be writing to the console if you close the handle and Attach/Alloc Console, right? Not writing to the redirected file (which is what I want).Bill Ticehurst