PInvoke errors calling external SetWindowsHookEx and GetModuleHandle

Question

I am trying to set windows hooks in my program to an external EXE. This will be used to monitor resizing/minimizing of the window, so I can resize my program similarly, docking to the window.

How do I get around error codes 1428 and 126 below?

When calling SetWindowsHookEx with a null hMod, I was getting this error 1428. I get the same error if passing the current module (instead of IntPtr.Zero), which it seems to get correctly, as so:

IntPtr module = PInvoke.GetModuleHandle(null);
[...]
SetWindowsHookEx(...,...,module,...);
int error = PInvoke.GetLastError();

1428 = Cannot set nonlocal hook without a module handle

I also tried to grab the external program I'm hooking as a module using GetModuleHandle:

IntPtr module = PInvoke.GetModuleHandle("communicator.exe");
int error = PInvoke.GetLastError();

But error is then set to:

126 = The specified module could not be found.

I am using the following PInvoke statements:

[DllImport("kernel32.dll", CharSet = CharSet.Auto, SetLastError = true)]
public static extern IntPtr GetModuleHandle(string lpModuleName);

[DllImport("user32.dll", SetLastError = true)]
public static extern IntPtr SetWindowsHookEx(HookType hookType, HookProc lpfn, IntPtr hMod, uint dwThreadId);

This is the procedure that is having the issue:

public void Install(IntPtr hWnd)
    {
        uint threadId;
        uint processId;

        if (hWnd == IntPtr.Zero)
        {
            threadId = (uint)AppDomain.GetCurrentThreadId();
            throw new Exception("Lync thread not found!");
        }
        else
        {
            threadId = PInvoke.GetWindowThreadProcessId(hWnd, out processId);
        }

        //IntPtr module = PInvoke.GetModuleHandle(null);
        //IntPtr module = PInvoke.GetModuleHandle(GetType().Module.FullyQualifiedName);
        IntPtr module = PInvoke.GetModuleHandle("communicator.exe");
        int error = PInvoke.GetLastError();

        m_hhook = PInvoke.SetWindowsHookEx(
            m_hookType,
            m_filterFunc,
            //Process.GetCurrentProcess().Handle,
            //threadId);
            //IntPtr.Zero,
            //module,
            //Marshal.GetHINSTANCE(
            //                                System.Reflection.Assembly.GetExecutingAssembly().GetModules()[0]
            //                        ).ToInt32()
            module,
            threadId);

        //IntPtr hinst = Marshal.GetHINSTANCE(Process.GetCurrentProcess().Handle);

        // http://msdn.microsoft.com/en-us/library/ms681385
        // ERROR_HOOK_NEEDS_HMOD - 1428 = Cannot set nonlocal hook without a module handle
        error = PInvoke.GetLastError();
    }

You left the important bits out but the error code is clear enough, you cannot set global hooks with C# code. — Hans Passant
What is the alternative? Do I have to write an external C++ DLL to handle it? — Shawn
Also I thought providing a Thread ID to SetWindowsHookEx makes it a Thread Hook--not a global hook--so I thought it would be allowed. — Shawn
The module handle has to be a handle to a DLL loaded into your process. That DLL must contain the filter function (an unmanaged function). Windows will then load that DLL into each process in the system and call the filter function in each process as required. So yes it needs to be in a C/C++ DLL to work. — shf301
"Windows will then load that DLL into each process in the system and call the filter function in each process as required." Oh okay, that makes sense, good explanation — Shawn

Mike Zboray Mike Zboray · Accepted Answer · 2011-10-15T02:10:37

You can't use GetModuleHandle for an external process. It must be a module that has been loaded into the current process.

PInvoke errors calling external SetWindowsHookEx and GetModuleHandle

2 Answers