Reverse engineering global hot key block

4
votes

I'm trying to reverse engineer (OllyDbg) an application (game) that disables (captures/intercepts/blocks) all global hot keys while its window (D3D) is active (except Ctrl-Alt-Del).

My question is in which ways could such application capture/disable global hot keys (including the winkey, CTRL+Esc, and global hotkeys set in other applications) while its window is active?

It seems to use DirectInput (if that matters). I see that it doesn't set any low level hooks (SetWindowsHookEx). In which other ways could such a thing it be done?

Observations made:

  • When the application is suspended (while its window is still in focus) the hot keys are still disabled while the window is in focus. Switching window to something else enables them. Switching back to the suspended applications window disables the hot keys again.
  • If doing the same thing but suspending the application after the window lost focus, and then switching back to the suspended window, hot keys are fully working.
  • Edit: The application appears to block global hotkeys set with RegisterHotKey, but does not disable any hotkeys made with help of a hook (SetWindowsHookEx).

The experiment above seems to indicate that it's some setting related to the window, that works even tho the app is suspended. Could it be some kind of DirectInput setting? (Altho the only DINPUT API function call I see is DINPUT8.DirectInput8Create.)

I'm doing this in hope of changing this behavior to make the hot keys from other applications work as usual while the games window is focused. Any ideas and tips appreciated.

1
c++winapireverse-engineeringhotkeysdirectinput
You can't disable all hotkeys - Ctrl-Alt-Del is always intercepted by the system. – Kerrek SB
@KerrekSB: Not necessarily. – josh3736
you might find the windows portion of the Doom 3 source of interest: github.com/TTimo/doom3.gpl/tree/master/neo/sys/win32 (funnily enough they disable task manager there as well, both ctrl+alt+del & crtl+shift+esc) – Necrolis
@Necrolis, good link, thanks. Atho they use SetWindowsHookEx to capture and disable those hotkeys, which OllyDbg tells me is not present in this case. But maybe it's/I'm just not finding that call for some reason (I am pretty novice at this). (How could I find it if the debugger doesn't list it? Hook it in some way?) – Qtax
I'd look at IDirectInputDevice8::SetCooperativeLevel(). The DISCL_EXCLUSIVE and DISCL_NOWINKEY flags are promising. – Hans Passant

1 Answers

1
votes

Exactly as Hans Passant suggested in the comments, it was DirectInput causing this behavior, due to the DISCL_EXCLUSIVE flag set with IDirectInputDevice8::SetCooperativeLevel.