How can I simulate mouse clicks by posting events to the Qt event system?

Question

I would like to do a rudimentary automation test of my Qt application. It records mouse events and writes them to a file (f.e. mousepress(300, 400)). When starting the automation, it reads the coordinates from the file, sends the appropriate mouse events and would do a pixel comparison with a previously saved screenshot.

Currently, I have an overlay widget that spans the app and has transparent mouse events. All it does is track the coordinates. When reading the data back in, that overlay paints a rectangle on the mouse press location. I need help when sending mousePressEvents to Qt's event system. It draws the points on the correct location but never actually does the physical click. Is there a way to do this with Qt or would I have to use Window's SendInput()?

Is there a way to pause and wait until the mouse event has finished? I would need to know when the event is complete in order to start the pixel by pixel comparison.

Widget::Widget( QWidget *parent )
: QFrame( parent )
, mPoint(QPoint(0,0))
{
   setWindowFlags(Qt::WindowStaysOnTopHint);
   setStyleSheet("background-color: rgba(0, 0,255, 2%);");
   setAttribute(Qt::WA_TransparentForMouseEvents, true);
   setGeometry(parent->geometry());
   ...
}

void Widget::run()
{
   QFile file( "coordinates.txt", NULL );
   if(!file.open( QIODevice::ReadOnly ))
       return;

   QTextStream in(&file);
   int i = 0;
   while (!in.atEnd())
   {
       QString line = in.readLine();
       if(line.startsWith("mousepress"))
       {
          int startIndex = line.indexOf('(');
          int endIndex = line.indexOf(')');

          QString coord = line.mid(startIndex+1, line.size() - startIndex - 2);
          QStringList nbr = coord.split(',');
          mPoint = QPoint(nbr[0].toInt(), nbr[1].toInt());
          QWidget *receiver  = QApplication::widgetAt(mPoint);
          QMouseEvent *event = new QMouseEvent(QEvent::MouseButtonPress, mPoint, Qt::LeftButton, Qt::LeftButton,  Qt::NoModifier);
          QCoreApplication::postEvent(receiver, event); // same result with sendEvent() 
          QCoreApplication::processEvents();
          update();
          // wait till the event finished, then continue with next point
      }
   }
}


void Widget::paintEvent(QPaintEvent *event)
{
  QPainter p( this );
  QPen pen;
  pen.setBrush(Qt::NoBrush);

  if(!mPoint.isNull())
  {
    pen.setColor(Qt::red);
    pen.setWidth( 2 );
    p.setPen(pen);

    p.drawRoundRect(mPoint.x(), mPoint.y(), 10, 10, 25, 25);
    p.drawText(mPoint, QString::number(mPoint.x()) + ", " + QString::number(mPoint.y()));
  }
}

[Edited]

I followed ddriver's suggestion and it works after a few changes: I save global and local positions in the file, to send to the QMouseEvent.

How could I be sure that the mouse click is complete before doing a screenshot and comparing it to a saved image?

void Widget::DoStep()
{
  if(!mInStream.atEnd())
  {
      QString line = mInStream.readLine();
      if(line.startsWith("MouseButtonPress"))
      {
          QPoint pos = parseGlobalPos();
          QPoint localPos = parseLocalPos();
          QWidget *receiver  = QApplication::widgetAt(pos);

          QMouseEvent *event = new QMouseEvent(QEvent::MouseButtonPress,localPos, pos, Qt::LeftButton, Qt::LeftButton,  Qt::NoModifier);
          QApplication::postEvent(receiver, event);
          QMouseEvent *eventRelease = new QMouseEvent(QEvent::MouseButtonRelease, localPos, pos, Qt::LeftButton, Qt::LeftButton,  Qt::NoModifier);
          QApplication::postEvent(receiver, eventRelease);

          // after successful click, take screenshot and compare them
      } 
   }

  if (!mInStream.atEnd())
      QMetaObject::invokeMethod(this, "DoStep", Qt::QueuedConnection);
  else
      file.close();
}

Answer 1

If I understand the problem correctly, its source is the blocking while loop, which blocks the thread and doesn't allow the event loop to spin and process events. There is no way to "pause" as that would block the event loop as well, and it wouldn't be able to do the work either, but there is a way to split the work to be done one step at a time, one event loop cycle at a time.

The solution is to not use a blocking while loop but implement an event system driven loop. Then you process one line on every event loop cycle until you run out of work.

Move the file, stream and all that stuff outside of the function, make them class members so they persist. Then in run() you simply setup the input for reading, and move all the event posting stuff to a new function, for example doStep(), and in run() after you setup you have a:

QMetaObject::invokeMethod(this, "doStep", Qt::QueuedConnection);

In doStep(), aside from the event stuff, at the end you also do scheduling:

if (!in.atEnd()) QMetaObject::invokeMethod(this, "doStep", Qt::QueuedConnection);
else // we are done, close file, cleanup, whatever

This way only one step will be executed per event loop cycle, allowing the event loop to spin and process the events. While there is work, a step will be scheduled for the next event loop cycle. You can also use a single shot timer to do this, or even a queued connection, which you can trigger by emitting overloading the event to emit a signal on completed. You do not, and should not force events to be processed, using this approach there will be no need to.

The concept in general is about a non-blocking async event driven self scheduling worker, I have posted a fully implemented one here. The upsides are you can track progress, pause, cancel and all that good stuff.

Answer 2

Try sending event to QGraphicsView's viewport:

qApp->sendEvent(view->viewport(), &mousePressEvent);