Automation tool similar to Sikuli

Question

I really like the feature in Sikuli that allows you to select a screenshot to click and specify where within that screenshot to click. For example when there are multiple buttons that look the same, just taking a screenshot of the button itself won't cut it because Sikuli can't differentiate.

Some other automation tools don't allow you to select where you want the mouse to click within a screenshot, or it is difficult to specify where you want the mouse to click.

So my question is, is there another software tool that has this feature that Sikuli has? The reason I'm looking for something else is because I don't want to rely on image recognition completely. I was hoping to use pywinauto with Sikuli to do some of the automation, but that isn't possible.

Can you please elaborate on your problems with using Sikuli are. Perhaps there is a way to overcome some of them without using other tools. Generally speaking, probably you won't be able to have the best of both worlds. Sikuli's biggest strength is in allowing you to interact with application that do not provide an API. On the other hand, it might be somewhat unreliable in some instances. — Eugene S
It's the unreliability that I'm trying to get around by minimizing my use of Sikuli. I have done some automation in Python using pywinauto.application to interact with objects but ran into the issue of objects not existing for some of what I want to click on, so Sikuli was the solution for that. What I want to do is still use Python whenever I can and use Sikuli image recognition only for clicking buttons, etc., where I can't use Python. Does that explain my problem better? — sushimama
Which OS'ses do you need? Propriety allowed? If windows-only freeware AutoIt may help, though it is very much outside the Python world. — Bookeater
Right now just Windows and yes proprietary allowed. I'm looking into anything I can at this point. I'd prefer to stick to Python if possible but that's not a requirement. — sushimama

Bookeater Bookeater · Accepted Answer · 2016-07-14T14:17:49

There is at least one: eggPlant by TestPlant.

It is an automation tool using image recognition, and

You can search part of the screen by setting the SearchRectangle global property.

References:
http://www.testplant.com
http://docs.testplant.com/ePF/using/epf-finding-images.htm#Working

For Python integration Automa would be nice, and it does have nifty image recognition functions where you can have multiple image parts working together. Not exactly what you are asking but it just might be of use.

References:
http://www.getautoma.com
http://www.getautoma.com/features/image_recognition

The odds are a bit longer but possibly AutoIt can help you as well. It interfaces with the standard Windows GUI objects and has the benefits of being both very stable and quite easy to use. It has a small tool you can use to show the properties of the screen object you want to access. No learning curve to speak of, however no image recognition or real integration to Python either.

Reference:
https://www.autoitscript.com/site/autoit/

Automation tool similar to Sikuli

2 Answers