It is available for dependency download in projects and this is the index entry.
If you used the snapshot version until now, just remove the -SNAPSHOT from the version in the dependency specification.
The OSSRH repository entry is not needed anymore, until you decide to again use snapshots of version 1.1.1 towards end of this year.
<dependencies> <dependency> <groupId>com.sikulix</groupId> <artifactId>sikulixapi</artifactId> <version>1.1.0</version> </dependency> </dependencies>
The observe feature (see docs especially for the usage with Java) allows to wait for an image to appear or vanish or to wait for changes of the pixel content in a region on the screen. This observation can be done inline (the script waits until the observe ends) or in parallel to the continuing workflow, meaning that the observation is delegated to a thread (hence runs in background).
Before running the observation, you register one or more events (appear, vanish or change) with the region, where you expect the events to happen. Since the beginning of the observe feature in Sikuli, you have to specify a handler function (callback), that is called, when the event happens and where you could handle the situation with the option to stop the observation at this point. With one region object (you can have different objects for the same area on the screen though) only one observation can be run at any time, but you might specify as many events as needed (… or what makes sense).
Here I only want to talk about background observes, which is the more interesting situation (the inline observe is only some bundled waits just a bit more sophisticated).
So the usual pattern in a Python script is:
def handler(event):
# do something
reg = <some region object>
reg.onAppear("someImage.png", handler)
reg.observeInBackground(FOREVER) # observe in background
# script immediately continues here
In the moment someImage.png appears in the given region, we want to do something (which is done in the handler function). Since the observe runs in background, our script continues and the handler does its job in parallel to our script in the moment the event happens.
When the script ends all running observations are stopped automatically.
With the revision in 1.1.0 the observe feature now is very reliable and produces valuable debug information if needed.
These are the major changes and new features:
Another new helpful feature, is the fact, that mouse actions are now handled like transactions: even in parallel workflows (like with a main script and background handlers) only one compound mouse action can be processed at any one time. This means a mouse click in the handler is completed before a parallel click in the main workflow will be processed and vice versa. This rather complex feature is worth its own article.
The same example as above, now continuing the observation after the event happened:
def handler(event):
# do something
event.getMatch().highlight(2)
if event.getCount() < 5:
event.repeat(1)
reg = <some region object>
reg.onAppear("someImage.png")
reg.observeInBackground(FOREVER) # observe in background
# script immediately continues here
Besides showing the getters it is principally the same as above. But now we limit the appearences to 5 (after that the event is deactivated) and say that the observation for this event should continue only after a pause of 1 second after returning from the handler.
With this last example I want to show the usage of named events.
The challenge is to scroll a webpage until an image gets visible. I know, that this can be rather easily done with exists() too, it is here done with observe for demonstration only.
For this snippet to work, the mouse must be positioned at a place, where the focused window accepts mouse wheeling.
reg = <some region object> imgStop = "imgStop.png" # the stop image to wait for stopAppeared = reg.onAppear(imgStop) # no handler, saving the name reg.observeInBackground(FOREVER) # start observation while not reg.isObserving(): wait(0.3) # hack currently needed while reg.isObserving(): # until observing is stopped wheel(WHEEL_UP, 10) # wheel (might be WHEEL_DOWN on Windows) wait(1) m = r.getEvent(stopAppeared).getMatch() # get the event's match hover(m) m.highlight(2)
We register an appear event without specifying a handler, but store the event’s name in stopAppeared. We use Region.isObserving(), to wait for the observation to stop because our one event has happened. While the observe is running, we use the mouse wheel to scroll the page. After the observation has ended, we get the event’s match to move the mouse there and add a 2-seconds-highlight.
The mentioned hack after starting the observation is currently needed, to bridge the startup time for the observation (loading the image and preparing the threaded search loop). This will be revised with version 1.1.1.
Another possibility would have been to use the features Region.hasEvents() or Region.hasEvent(stopAppeared) instead of Region.isObserving() like this:
while len(reg.hasEvents()) == 0: # returns a maybe empty list # or while not reg.hasEvent(stopAppeared): # returns the event or None
Hope this gives some ideas about new possibilities with version 1.1.0
… more Information and how to get started on the SikuliX homepage.
The next days I will talk here about new features and major changes.
A taiwanese lover of the SikuliX API created a visual automation tool on Android, that was inspired by the SikuliX features. The app runs directly on the device and the scripting itself has to be done with the scripting language Lua.
The app name AnkuLua is based on: Android Sikuli Lua.
The entry on Google Play says:
Let AnkuLua run application (like playing game) for you.
✓no root required
✓one script for all devices
✓use simple Lua script language
✓straightforward usage
✓fast image matching
✓auto click/tap
with AnkuLua, users can do followings, but not limited to
✓click on pictures (with offset)
✓wait for pictures to appear in specified time
✓wait for pictures to vanish in specified time
✓type texts
✓sent key event (like home, back)
✓drag and drop from one picture to another one
✓set similarity to compared pictures
✓search only some regions of the screen
According to the developer: the implementation is mainly written in Java, utilizes OpenCV and running Lua scripts is supported by a service. It is planned to make AnkuLua more robust and stable as the next steps.
The app on Google Play: AnkuLua
SikuliX 