Capturing Window Events in Linux/Ubuntu (Screen-Capture + Mouse/Keyboard-Events)

Question

I want to develop an Application in C++ for Linux that needs to interact with other programs via standard IO (Mouse + Keyboard). It should be able to send Mouse/Keyboard Input to a Window and to capture a "screenshot" of that specific window.

(1) Now I have done some research and I know that Linux uses the "X Window"-System. Is it advised to start on this layer for the programming or should I use some higher level framework (e.g. wxWidgets)?

(2) Is it possible to send input events to/capture the screen of a window even though it's not focused?

(3) I am not asking you for code, but I would love to work through some literature on that topic. Sadly, I couldn't find any good sources on this topic.

It would be great if someone could help me on this one!

Thank you in Advance!!

[ Note: This program should running as backend and later execute commands from a java application. The plan is to implement this backend C++ App for Windows/Linux seperately ]

Andrzej Pronobis Andrzej Pronobis · Accepted Answer · 2015-05-07T19:49:55

Solution 1 I can recommend a piece of code for sending keyboard events which I use myself to simulate key presses. It is based on XSendEvent.

#include <X11/Xlib.h>
#include <X11/keysym.h>

/** Modifier states */
#define MOD_ALT 0x8
#define MOD_CONTROL 0x4
#define MOD_CONTROL_ALT 0xc
#define MOD_SHIFT_CONTROL 0x5
#define MOD_SHIFT_ALT 0x9

XKeyEvent createKeyEvent(Display *display, Window &win,
                           Window &winRoot, bool press,
                           int keycode, int modifiers)
{
   XKeyEvent event;

   event.display     = display;
   event.window      = win;
   event.root        = winRoot;
   event.subwindow   = None;
   event.time        = CurrentTime;
   event.x           = 1;
   event.y           = 1;
   event.x_root      = 1;
   event.y_root      = 1;
   event.same_screen = True;
   event.keycode     = XKeysymToKeycode(display, keycode);
   event.state       = modifiers;

   if(press)
      event.type = KeyPress;
   else
      event.type = KeyRelease;

   return event;
}


void pressKey(Display* display, Window &win_focus, Window &win_root,
              KeySym key, int modifiers)
{
    // Send a fake key press event to the window.
    XKeyEvent event = createKeyEvent(display, win_focus, win_root, true, key, modifiers);
    XSendEvent(event.display, event.window, True, KeyPressMask, (XEvent *)&event);

    // Send a fake key release event to the window.
    event = createKeyEvent(display, win_focus, win_root, false, key, modifiers);
    XSendEvent(event.display, event.window, True, KeyPressMask, (XEvent *)&event);
}

Then, you can use it to send a key event (in this case Alt+W) as follows:

// Obtain the X11 display.
Display *display = XOpenDisplay(0);
if (display == NULL)
{
  printf("Null display!\n");
  return 1;
}

// Get the root window for the current display
Window win_root = XDefaultRootWindow(display);

// Find the window which has the current keyboard focus
Window win_focus;
int    revert;
XGetInputFocus(display, &win_focus, &revert);


pressKey(display, win_focus, win_root, XK_w, MOD_ALT);

if (display)
  XCloseDisplay(display);

Solution 2 A similar behavior can be achieved using the XTest library. It operates on a "higher level" though and for instance the window manager will pick up those keypresses as well. The pressKey function for XTest can look like this (not very clean code):

void pressKey(Display* display, KeySym modifier1, KeySym modifier2, KeySym key)
{
    // Release every other modifier
    KeyCode keycodec = XKeysymToKeycode(display, XK_Control_L);
    KeyCode keycodea = XKeysymToKeycode(display, XK_Alt_L);
    KeyCode keycodes = XKeysymToKeycode(display, XK_Shift_L);
    XTestFakeKeyEvent(display, keycodec, False, 0); // key release event
    XTestFakeKeyEvent(display, keycodea, False, 0); // key release event
    XTestFakeKeyEvent(display, keycodes, False, 0); // key release event
    XFlush(display);

    // Press the actual keys
    KeyCode keycode1 = XKeysymToKeycode(display, modifier1);
    KeyCode keycode2 = XKeysymToKeycode(display, modifier2);
    KeyCode keycode3 = XKeysymToKeycode(display, key);
    if (keycode1)
        XTestFakeKeyEvent(display, keycode1, True, 0); // key press event
    if (keycode2)
        XTestFakeKeyEvent(display, keycode2, True, 0); // key press event
    if (keycode3)
    {
        XTestFakeKeyEvent(display, keycode3, True, 0); // key press event
        XTestFakeKeyEvent(display, keycode3, False, 0); // key release event
    }
    if (keycode2)
        XTestFakeKeyEvent(display, keycode2, False, 0); // key release event
    if (keycode1)
        XTestFakeKeyEvent(display, keycode1, False, 0); // key release event
    XFlush(display);
}

Solution 3 Another solution is to just script your way out with such tools as xdotool. Basically, most of what you described can be achieved with a bash script. Also xdotool source code is a great source of information on how to achieve all you want.

Capturing Window Events in Linux/Ubuntu (Screen-Capture + Mouse/Keyboard-Events)

1 Answers