If you use a framework like Electron to develop a desktop version of your web application, you may have run into a common problem when testing your application:
Most automated testing tools for web applications—such as Selenium or Cypress—can’t test anything outside of a single tab within a single web browser.
So while you can test most of the critical functions of your app in the web version, there are some critical test cases for the desktop version that you simply can’t automate (such as setting up desktop notifications or verifying the app can be downloaded and installed).
Your options are to use an additional tool just to test those desktop-specific use cases, or to test them manually. In either case, testing the desktop app can slow down the release process, and it often gets neglected altogether.
Our automated testing platform, Rainforest QA, solves this problem.
Rainforest is a codeless automated testing platform that offers a complete solution for testing web and desktop applications. Unlike most software testing tools, Rainforest tests interact with applications directly in the user interface, just like a human user would. This means Rainforest tests can interact with the operating system, open and interact with desktop apps, manipulate desktop settings, and more.
In this article, we’ll show you how to use Rainforest QA to automate all critical test cases of a desktop application built with Electron—so you can stop juggling multiple tools. And, the best part is that Rainforest doesn’t require programming skills to use, so anyone on your team can get started in minutes.
Table of Contents
- Rainforest QA: No-code Automated Testing for Any Functional Test Case
- Writing and Maintaining Tests is Easy for Anyone
- Quickly Understand Every Test Result with Video Replays
- Access a Worldwide Community of Experts
- Get Free Testing Every Month—Then Only Pay for What You Use
- How Rainforest Compares to Four Other Desktop Application Testing Tools
Sign up for Rainforest QA—you can run five hours of free no-code automated functional tests for desktop applications every month. It’s only $5/hour after that.
Rainforest QA: No-code Automated Testing for Any Functional Test Case
Before we get any further, let’s get clear on what type of testing we’re talking about. Whenever we talk about testing in this article, we’re referring to functional testing, and even more specifically, regression testing, because this is the type of testing that’s most suitable for automation. We’re not going to cover security testing, usability testing, performance testing, or compatibility testing because those tests are typically run less frequently than functional tests, which must be re-run with every release.
Most software testing tools search for element locators in the underlying code of an application to interact with and verify elements of the application. This limits test steps to only interacting with the application itself and not with the browser, browser settings, or desktop functions.
Rainforest QA, on the other hand, takes an entirely different approach that allows you to automate test steps that interact with anything visible on your screen.
Rainforest tests use pixel-matching to locate and interact with any elements that appear on screen. This is essentially what real users do—they interact with the visual layer without touching the underlying code. And, Rainforest QA automates operating systems on virtual machines rather than automating single browser tabs like most tools.
All of this means you can automate nearly any test case—such as downloading and installing an application or extension, changing browser or desktop settings, etc.—for both the web version and desktop version of your application. And, you’ll be testing as close to the real user experience as possible.
Here are a few more ways that Rainforest QA makes it really easy to improve your test coverage:
Writing and Maintaining Tests is Easy for Anyone
With Rainforest QA, the process for creating a test for a desktop application is the same process as creating a test for a web application. To write or edit a test step in Rainforest QA, you choose from a dropdown list of actions (like “Click” and “Fill”) and take a screenshot of the area you want to apply the action to. You can preview each step as you go and/or preview the entire test at the end to ensure it will do what you intended it to.
When you go to run the test, the pixels of each screenshot are compared against all combinations of pixels on the screen. If a match is found, the test continues or is marked as successful—otherwise the test fails.
If a test needs to be updated because of some change in the UI of the application, Rainforest makes maintenance fast and easy with embedded tests. You can turn any sequence of test steps into an embedded test that you can drag-and-drop into other tests. If you make a change to the original sequence of steps, that change will be reflected in every other test that uses that embedded test.
For example, you could turn your login sequence into an embedded test, like we’ve done in the example below:
Without embedded tests, if you find a bug in a login sequence, for example, you would have to fix the bug in every test that uses that login sequence. With embedded tests, you only have to fix the bug in one place and it gets applied across the board.
Additionally, you can choose to run any test on virtual machines with macOS and Windows OS—without any additional setup. You can also choose between current and older versions of any of the four major browsers (Safari, Microsoft Edge, Chrome, and Firefox).
To run a test or a set of tests, you can manually start the run within the Rainforest platform. Or, your dev team can kick it off within your CI/CD pipeline using our CircleCI Orb, GitHub Action, API, or CLI.
Quickly Understand Every Test Result with Video Replays
Understanding why a test failed is often one of the most time-consuming parts of software testing. Video replays can make it go a lot faster, but most tools are limited in how many runs they’ll let you record and how long you can keep the recordings. Additionally, a lot of tools will make you set up the video recordings manually instead of automatically recording test runs for you.
Rainforest QA automatically records every test—whether it passes or fails—and lets you keep a history log of video recordings indefinitely. This lets you compare a failed test run to a successful test run to see what changed. It also lets you see how the failure would’ve appeared to end users so you can prioritize which bugs to fix first.
Rainforest QA also provides HTTP logs and browser logs for each test, which can help you identify whether or not the test failure was due to hiccups in the test environment rather than a bug in the software. By clicking the “investigate action” button under any test step, you can view even more details about how the result was determined.
Finally, if there is a real bug, Rainforest QA helps you streamline communication between whoever is evaluating the test results and the dev team with a Jira integration.
Anyone can automatically create a Jira ticket that includes the failed test steps, a screenshot of the failed test step, HTTP logs, and a link to the full test results and video recording in Rainforest. This allows the software development team to fix bugs with less disruption to their workflow.
Access a Worldwide Community of Experts
While Rainforest QA lets you automate most desktop application test cases, there will always be some tests that require human interpretation, such as a test case with a CAPTCHA. These tests are better suited for manual testing.
Many teams find that Rainforest QA improves the speed of their testing process so much that these few test cases are easy to handle in-house. However, Rainforest QA also gives you access to our community of worldwide QA experts for the fastest manual testing available—on average, results get returned within 17 minutes.
Any Rainforest test can be automatically converted into a plain-text test script for our manual testers to follow.
Or, you have the option to write a plain-text test script from the start.
Our community of manual testers is also a useful option for test cases that experience frequent change (and would therefore cause automated tests to constantly break)—like features that are still in beta, or features that involve user generated content.
Free Testing Every Month—Then Only Pay for What You Use
Our Professional plan makes web and desktop application test automation accessible to anyone. This plan has everything you need for test automation—including parallel testing on virtual machines and unlimited team members—without any hidden costs.
You get 5 hours of free testing every month and it’s only $5/hour after that.
The Professional plan also includes a 14-day trial of manual testing by our community of QA specialists. After that, the manual testing service is just $25/hour.
For both automated and manual testing, you only pay for the tests you run. It doesn’t cost anything to write/edit/preview tests, view test results, or otherwise manage your test suite.
And unlike many automated testing tools, our Professional Plan doesn’t require a contract. This means you can scale your testing up or down from one week to the next to meet project requirements at a moment’s notice while only paying for what you use.
How Rainforest Compares to Four Other Desktop Application Testing Tools
Most tools that offer solutions for desktop application testing are limited in functionality and operating system compatibility. In this section, we provide a brief breakdown of how Rainforest QA compares to four common test automation tools. Each tool represents a different type of testing solution for desktop application testing.
Katalon: Record-and-Playback Tool Built on the Selenium WebDriver
Record-and-playback tools can make it easier and quicker to write simple test cases. However, they still only interact with the underlying code of the application to perform test steps—which means each test case is locked into one browser tab or desktop window.
Not all record-and-playback tools offer desktop testing, but some, like Katalon, do offer desktop testing. (Note: Selenium IDE is another popular record-and-playback tool, but it’s built for web applications, not desktop applications.)
Katalon’s automation framework for testing desktop applications focuses heavily on creating a solution for Windows OS. This means that most of their helpful features, such as built-in keywords, are only available for Windows OS. They let you run tests locally for macOS and Linux, however, if you want to access their remote testing grid to run multiple tests at once, it can only be done with a Windows OS.
As we mentioned earlier, Rainforest QA lets you test anything visible on your screen on both Windows OS and macOS. And, you can run multiple tests at once for either operating system.
If you’d like to read a more detailed comparison of Katalon vs. Rainforest’s test automation for web apps, read this article.
SikuliX: Automated Image Recognition Software Testing
SikuliX is an open-source solution that offers UI testing using image recognition powered by OpenCV to identify and interact with UI elements. This is one of the only solutions that can interact with anything on your screen just like Rainforest QA. However, SikuliX does require a software engineer to write tests in a programming language.
Additionally, SikuliX has to be downloaded onto your desktop. If you need to verify that a particular feature of a desktop app works without an internet connection, a tool like SikuliX would be helpful because it runs tests locally.
It is compatible with macOS, Windows OS, and Linux OS which makes it more flexible than a lot of desktop application testing solutions—other than Rainforest QA, which offers many configurations of OS and browsers.
Finally, like a lot of open-source tools, SikuliX offers limited documentation and support.
Applause: Manual Software Testing Service
Instead of automating desktop application tests, some QA testing teams choose to outsource their manual testing. Solutions like Applause will take over some, or all, of your manual testing for you. This means they write/edit the tests, decide when to kick off a test run, and triage test results for you. Most of these manual testing services also provide options for testing mobile applications.
We’ve written in more detail about the downsides of most manual software testing services, but essentially it comes down to how quickly you can get results and how much say your team has over what gets tested and when.
Rainforest QA provides the fastest manual testing currently available—we provide results in 17 minutes on average from the time you request the test run—and gives you complete control over all aspects of your testing.
Winium: Automation Library for Desktop Apps
There are various libraries available for different programming languages that were developed to help you test desktop applications on various operating systems.
For example, Winium is a Selenium WebDriver implementation that lets you write tests in your preferred scripting language—such as JavaScript with node.JS, Java, PHP, etc.—for testing Windows desktop applications built on WinForms and WPF platforms. And, it’s compatible with nearly any testing framework.
However, these automation libraries aren’t a stand-alone testing solution, so you’ll have to manage and learn how to use multiple tools. And, once again, you’ll be entirely reliant on software engineers for all testing, which tends to slow down the release process.
Ready to fix the coverage gaps in your desktop app testing? Sign up for Rainforest QA—you get five hours of no-code UI automation testing for free, every month. It’s only $5/hour after that.
