How to Get Started with Sikuli for Test Automation

Test Automation is the way to improving the efficiency and effectiveness of enterprise software delivery. While manual testing has its place, humans are slow and error-prone. If you’re looking to progress from a reactive QA shop to a modern proactive and predictive quality assurance organization, you have to go all-in on test automation. Here’s how to get started with Sikuli for test automation.

What is Sikuli?

Sikuli is an image recognition tool that can detect any image on your computer screen as long as it is visible to the human eye. It can then perform actions on the detected images, which makes it ideal for replicating human interface testing. It can perform repetitive “human” actions and interface with software as a user would.

Why use Sikuli for test automation?

Consider an example of automating the functional tests for a web application. The first test tool that would come to our minds would be Selenium for its obvious benefits of being free and a lot of support as it is widely used.

However, when you start exploring the functionality of the application you figure out that your product is primarily a video sharing website. A lot of the content on the website are videos which are previewed using the flash player. It is not possible to detect the flash player elements, play the videos and validate video playback using Selenium.

One possible strategy to test your application would be to eliminate automating all the test cases related to video playback and validation. However, when you do that you are left with very few test cases that can be automated. The bulk of the test cases would have to be run manually. Since the benefits of introducing automation are almost negligible in this scenario it becomes very difficult to convince the stakeholders to use automation testing as a strategy.

Here is where Sikuli comes to our aid with its image detection capabilities. As I have already mentioned, Sikuli can detect and perform actions on anything it can see on the screen. So, as long as your site opens up on your machine and you can see that flash video playing on your screen Sikuli also can see it. All you need to do now is build an automation framework around the Sikuli APIs to automate most of the functional test cases.

Check out our article on Predictive Quality Assurance for Software Engineering >

What are the other benefits of Sikuli?

Well, anyone who has had experience in automating tests for any kind of system has always come across scenarios where in spite of the system has a lot of exposed interfaces (like rest APIs, command-line interfaces), there would be some tests which would not be automatable as the system does not have an exposed interface to do so. So yes, this has to be one of the major benefits of opting for Sikuli as a tool for your automation activities.

But there are other advantages of Sikuli as well,

  1. Free: Yes, Sikuli is open-sourced and you do not have to burn a hole in your pocket to get it.
  2. Cross-Platform: Sikuli does not run any code on the System Under Test. The Sikuli code runs on the Sikuli server (Windows or Mac, and yes the syntax for the code is the same on Windows and Mac). The Sikuli code server just needs to see the System Under Test on its screen and you are good to go.
  3. Remote Testing: Sikuli can perform testing of applications running on remote servers. Oh yes, and you do not need fancy hardware or software setup to do so. Just open up your System Under Test on the Sikuli server using Remote Desktop for Windows, VNC client or any other screen sharing application. Remember, all Sikuli needs are to see your application on the screen of the machine you are running Sikuli on.
  4. Integration with other tools: Sikuli can also be integrated with Selenium using the selenium web driver very easily. This allows you to leverage the advantages of Selenium along with the advantages of Sikuli.
  5. Ease of use: Sikuli has a very simple syntax and can be either written using the Java syntax (when used along with selenium java web driver or in case you just do not want to ditch java) using Eclipse or other Java IDEs or, you can use the SikuliX IDE to write Sikuli scripts which has a Pythonic syntax. I found the second one to be very helpful for newbies to get acquainted with the capabilities of Sikuli without much external help.

Check out our article on How to Determine Cost of Poor Quality in Software Engineering >

Sample Code using Sikuli

The following screenshot shows a simple Sikuli script used to open Gmail in Firefox with a little bit of error handling. And yes it is that simple:

See our article on How DevOps Improves Quality Assurance for Software Engineering >

Getting Started with Sikuli for Test Automation

Step 1: Go ahead and download Sikuli from here. You can also find detailed documentation to get you started up on Sikuli.

Step 2: You will need JRE installed on your system before you can install Sikuli as Sikuli is written in Jython.

Step 3: The installer is a jar file. Double click on this jar file to install Sikuli.

Step 4: Make sure you place the installer jar file in a location where you have the permissions to execute the jar file. The location of the installer is important because all the necessary Sikuli files will be installed in the same folder as the installer. I placed my file in D:\Sikuli.

Step 5: Here’s a screenshot of the folder where Sikuli is installed.

Step 6: Following is the screenshot of the Sikuli IDE.

See our Interview with a Leading Test-Automation Engineer >


Sikuli is a key weapon in modernizing your enterprise software testing approach. It allows you to replicate human interaction, improve efficiency, and receive more consistent results.


Explore other topics of interest