How to display a modal dialog window from another process?

Question

I have a 32-bit MFC application that uses a custom library that would be a nightmare to re-compile into x64. In general the app doesn't really need to run as 64-bit, except in one case -- and that is to render contents to display in a dialog window, which can benefit from a larger addressing space.

So my goal is to "imitate" CDialog::DoModal method but for a dialog in another process.

I built that dialog window as a standalone x64 MFC dialog-based application. It takes a file path as it's input parameter, does all the work internally, and returns simple user selection: OK, Cancel.

So I do the following from my main parent process:

//Error checks omitted for brevity
CString strCmd = L"D:\\C++\\MyDialogBasedApp.exe";

HWND hParWnd = this->GetSafeHwnd();

SHELLEXECUTEINFO sei = {0};
sei.cbSize = sizeof(sei);
sei.fMask = SEE_MASK_FLAG_NO_UI | SEE_MASK_UNICODE | SEE_MASK_NOCLOSEPROCESS;
sei.nShow = SW_SHOW;
sei.lpVerb = _T("open");
sei.lpFile = strCmd.GetBuffer();
sei.hwnd = hParWnd;

BOOL bInitted = SUCCEEDED(::CoInitializeEx(NULL, COINIT_APARTMENTTHREADED | COINIT_DISABLE_OLE1DDE));

ShellExecuteEx(&sei);

DWORD dwProcID = ::GetProcessId(sei.hProcess);

//Try to get main Wnd handle for the child process
HWND hMainChildWnd = NULL;
for(;; ::Sleep(100))
{
    hMainChildWnd = getHwndFromProcID(dwProcID);
    if(hMainChildWnd)
        break;
}

HWND hPrevParWnd = ::SetParent(hMainChildWnd, hParWnd);
if(hPrevParWnd)
{
    //Wait for child process to close
    ::WaitForSingleObject(sei.hProcess, INFINITE);

    //Reset parent back
    ::SetParent(hMainChildWnd, hPrevParWnd);
}

::CloseHandle(sei.hProcess);

if(bInitted)
    ::CoUninitialize();

where getHwndFromProcID is taken from here.

This kinda works, except of the following:

(1) There are two icons on the taskbar: one for my main app, and one for the child app. Is there a way not to show a child icon?

(2) I can switch focus from child window to parent and vice versa. In actual modal dialog window one cannot switch back to parent while child is open. Is there a way to do that?

(3) If I start interacting with the parent, it appears to be "hung up" and the OS will even show it on its title bar.

So I was curious if there's a way to resolve all these?

RbMm RbMm · Accepted Answer · 2016-12-27T02:37:17

you need pass pointer of own window to child process
you need process windows messages, while you wait on child process exit. WaitForSingleObject unacceptably here - need use MsgWaitForMultipleObjectsEx
child process must set your window as self owner window at create time - you not need call SetParent

with this all will be worked perfect. in your 32-bit MFC application you need use next code :

BOOL DoExternalModal(HWND hwnd, PCWSTR ApplicationName)
{
    STARTUPINFO si = { sizeof(si) };
    PROCESS_INFORMATION pi;
    WCHAR CommandLine[32];
    swprintf(CommandLine, L"*%p", hwnd);

    if (CreateProcessW(ApplicationName, CommandLine, 0, 0, 0, 0, 0, 0, &si, &pi))
    {
        CloseHandle(pi.hThread);

        MSG msg;

        for (;;)
        {
            switch (MsgWaitForMultipleObjectsEx(1, &pi.hProcess, INFINITE, QS_ALLINPUT, 0))
            {
            case WAIT_OBJECT_0:
                CloseHandle(pi.hProcess);
                return TRUE;
            case WAIT_OBJECT_0 + 1:
                while (PeekMessage(&msg, 0, 0, 0, PM_REMOVE))
                {
                    TranslateMessage(&msg);
                    DispatchMessage(&msg);
                }
                continue;
            default: __debugbreak();
            }
        }
    }

    return FALSE;
}

in MyDialogBasedApp.exe let use MessageBox as demo dialog. we will be use your MFC window as first argument to it.

void ExeEntry()
{
    int ec = -1;

    if (PWSTR CommandLine = GetCommandLine())
    {
        if (CommandLine = wcschr(CommandLine, '*'))
        {
            HWND hwnd = (HWND)(ULONG_PTR)_wcstoi64(CommandLine + 1, &CommandLine, 16);

            if (hwnd && !*CommandLine && IsWindow(hwnd))
            {
                ec = MessageBoxW(hwnd, L"aaa", L"bbb", MB_OK);
            }
        }
    }

    ExitProcess(ec);
}

with this code:

(1) There are only one icon on the taskbar for your main app

(2) You can not switch focus from child window to parent and vice versa. all work as actual modal dialog window

(3) parent not "hung up" because it processing windows messages (MsgWaitForMultipleObjectsEx) - your code is "hung up" because you not do this, but wait in WaitForSingleObject

How to display a modal dialog window from another process?

3 Answers