How do I get sikuli to wait until the mouse pointer changes from "busy" to "not busy?"

Question

I interact with a GUI application both locally and remotely (via a VPN). When I work with it locally, most actions happen pretty quickly and my sikuli scripts execute fine. However, when I work with it remotely, actions take longer to complete.

That is, Sikuli may detect the screen and elements to wait for, but the cursor is still "busy" (e.g. the spinning circle in windows). This results in my sikuli scripts breaking when working with the application remotely due to the increased "lag" (not sure if that's the right term for it).

For example, if I did

wait(laggy_field.jpg)
click(laggy_field.jpg)
type("so laggy")

Sikuli will wait for laggy_field. Laggy_field will appear but the cursor will still be busy. However, sikuli does not appear to be aware of this and moves the curosr to the field and clicks the field. The result is that the focus is not on that field because the cursor was actually busy/inactive. Thus, the type("so laggy") command results in keys being sent to nowhere.

If I put in an arbitrary time like 30 seconds to wait in between the wait and click commands the scripts will "work" but I would have liked a more elegant solution. It seems like the ability to detect changes in the mouse pointer is some way off Sikuli's roadmap

Answer 1

Not too sure what the question is, but here are a couple of suggestions:

(1) Are you sure your script is seeing the laggy_field?

wait(laggy_field.jpg)
hover(laggy_field.jpg)
click(laggy_field.jpg)
type("so laggy")

If the mouse does not move to the laggy_field, you need to increase the timeout of the wait() instruction. See below where I wait a max of 10s on laggy_field.jpg

wait(laggy_field.jpg, 10)

(2) If you insert an arbitrary wait time, does the click work?

wait(laggy_field.jpg, 10)
hover(laggy_field.jpg)
wait(10)
click(laggy_field.jpg)
type("so laggy")

(3) Try replacing the wait(10) with a wait on the correct mouse pointer (jpg taken from hover location within the laggy field box)?

wait(laggy_field.jpg, 10)
hover(laggy_field.jpg)
wait(laggy_not_busy_mouse.jpg, 10)
click(laggy_field.jpg)
type("so laggy")

Answer 2

The best way to get rid of script execution failure due to lag ,is to wait for a particular image pattern ,you can write something like this

while not exists('some_image.jpg'): wait ..perform next step

Note: this wait method doesn't have parameter once this condition is met you don't have to rely on explicitly mentioning timing using sleep(x) etc.

you'll need to identify for a specific image in your application which appears till the next step execution is not carried out. and accordingly you can base while not exists looping criterion in your sikuli script.

Answer 3

Sorry to grave-dig, but I came up with a class for making this simpler. It uses the exact() method to change the matching algorithm from "fuzzy" to a more precise algorithm.

class Element:
    def __init__(self, element, region):
        self.element = element
        self.region = region

    def stateChanged(self):
        if self.region.exists(Pattern(self.element).exact()):
            return False
        return True

Usage:

gmailEmailField = Element("1458930208197.png", Region(438,239,388,342))

while not gmailEmailField.stateChanged():
    wait(.5)

Just do this with each element in your application if it becomes non-responsive. That way you can make sure your elements are being filled in, or track when the application becomes responsive again since it uses exact matching.