Automating your software tests can speed up testing and provide more reliable test results. For most software teams, automation is necessary to achieve adequate test coverage.
If you’re just getting started with automated software testing, you may be overwhelmed by all the options and have a hard time figuring out which tools are right for your team.
In this post, we explain the three main types of testing tools—true no-code tools, record-and-playback tools, and coding frameworks for automated testing—and discuss which ones are best suited for teams just getting started with software test automation.
We’ll start with the easiest automation solution for anyone to use then move down the list toward the most difficult for beginners.
Note: Once you’ve chosen a tool, you can check out our post on how to build your test strategy.
Sign up for Rainforest QA—an all-in-one no-code automated testing tool that’s perfect for beginners and scales with you as your team and your product grow. Improve the effectiveness and efficiency of your testing without having to hire additional headcount. You can run up to five hours of no-code automated tests for free, every month. It’s only $5/hour after that.
Many of the most popular web automation frameworks like Selenium and Cypress require that you write code that the framework automates to interact with the underlying code of your application. But not all teams have engineers available to write and maintain these tests on a regular basis.
There are tools labeled codeless (or low code) that generate these lines of testing code for you (more on that in a later section) in a no- or low-code fashion, but eventually you’ll need to understand the code in order to understand why a test failed or to edit the test.
Very few tools are truly no-code. On the other hand, our tool, Rainforest QA, takes a completely different approach to testing web applications that allows for true no-code software testing.
Rainforest QA uses pixel-matching and preset actions to perform test steps—which means anyone, even a non-developer, can create, edit and run tests. Here’s how that works:
To create (or edit) each step in a test, you choose from a dropdown list of actions (e.g., click, fill, scroll, etc.). Then, click-and-drag to take a precise screenshot of the area you want to apply the action to.
When you run the test (which is as simple as a few clicks), the pixels of these screenshots are compared against all combinations of pixels on the screen to find a match. If a match is found, the test will perform the action or continue to the next step.
This means the visual layer and the functionality of your app will be tested simultaneously.
Then, when you’re ready to run a set of tests, your dev team can kick them off within your continuous integration pipeline using our CircleCI Orb, GitHub Action, API, or CLI. Or you can manually kick off a set of tests within the Rainforest platform.
You can also test things like sending and checking emails, activating new user accounts, and downloading files. We provide a library of test data with randomized values (email addresses, credit card numbers, names, etc.) so you won’t have to spend as much time preparing test user accounts.
One of the most time-consuming aspects of testing for many teams is figuring out why tests failed.
Rainforest QA automatically records a video of every test, whether it passes or fails. You can compare each test run to quickly see if anything in the UI changed between runs. Video replays also show you exactly how the failure would’ve appeared to a real user, which can help you decide how important it is to fix the failure.
When watching video replays, the test script will appear to the left of the video. Any step that fails is highlighted in red and can be looked at in greater detail by clicking the ‘investigate action’ button below it.
For each test failure, Rainforest also provides HTTP logs, browser logs, settings, and more. These details are often the key to identifying failures caused by hiccups in the test environment.
On Rainforest, your tests will run on our network of virtual machines, which can simulate over 40 combinations of browsers and platforms for both desktop and mobile applications.
As you build your test suite, test execution can quickly become a bottleneck unless it’s integrated into your team’s CI/CD pipeline. That’s why Rainforest QA offers a CLI and integrations with CI tools including CircleCI and GitHub.
Rainforest QA also offers:
To begin using automation to test a software application, you need a way to write tests, manage and organize the test suite, run tests, and understand why the tests failed.
Many tools will charge you extra for each of these features or won’t provide them at all.
Rainforest QA is an all-in-one platform which means you’ll have access to everything you need to create, execute, and manage as many tests as you need. You can start for free with up to 5 hours of testing per month. It’s only $5/hr after that for automated testing and $25/hr for manual testing with our community of QA specialists.
On our Professional plan, you won’t be locked into a contract. Instead, you only pay for what you use. You can easily scale up or down as needed to stay within your budget while meeting your testing needs.
If you think Rainforest could be the right fit for your team, sign up today.
To write a test with a record-and-playback tool, you manually interact with a user path in your application while the tool records your actions as a testing script. When you playback your recording, the tool just executes the testing code it generated. While this is a relatively easy way to create automated tests, it’s extremely limited in the kind of test data it can handle and the kind of user paths it can reliably test. And when these tests fail, you’ll have to refer to the testing code to understand what caused the failure and edit the test.
Most record-and-playback tools support cross browser testing and are an all-in-one solution.
Selenium is one of the most well-known software testing tools available. Although it was originally just a testing platform that automated interactions with browsers, Selenium now offers a web driver (which allows you to connect to additional testing language libraries) and a record-and-playback option. The record-and-playback option is called Selenium IDE.
Selenium IDE is the only open-source record-and-playback option in this list. Although you won’t have to pay for the service, the only support you’re offered is through their community chat and written documentation. You will also have to pay for a testing grid to run multiple tests at once.
Katalon Studio uses the Selenium WebDriver to write tests but was designed to support collaboration between developers and non-developers.
Unlike some record-and-playback tools, Katalon lets developers stop and restart a recording at any point during test creation or execution to insert additional lines of code. This makes it much easier to create complex test cases and fix broken tests. Katalon also provides helpful tools such as built-in keywords and project templates to help create test cases faster.
In addition to their record-and-playback feature, IBM Rational Test lets you take snapshots of the underlying code of your application at any given moment. These snapshots can be used to verify the state of your application before or after an action.
If you’re already using one of IBM’s many other business services, it may be useful to keep your services in one place. However, reviews suggest that the IBM user interface is more complicated to use than other testing software and was designed for technically advanced QA teams.
IBM runs on Windows and Linux operating systems.
For the test scenarios where you need to insert additional code, TestComplete offers keyword testing that lets you search for and insert actions based on preset keywords.
TestComplete also offers an object recognition engine which allows you to test elements inside a PDF or chart.
Ranorex Studio uses a mix of recorded actions and drag-and-drop actions from a preset list or repository for test creation. You can also take screenshots at any point during the test to verify elements of web apps.
Rather than running tests on a virtual machine like Rainforest QA, Ranorex tests are executed on real devices—either your own local devices or on remote Ranorex devices.
Unlike most tools, they offer four different tiers of perpetual licenses. Each one includes only one year of support.
Like Ranorex, Telerik Test Studio uses a mix of selecting preset actions and recording actions. Telerik puts more of an emphasis on building your tests with preset actions, so the tests may take longer to create than other record-and-playback options.
They offer the option to run tests on a headless browser meaning you won’t see the UI as the test runs. Instead, the tests will run the code via a command-line interface or network communication. This can make tests run faster because you don’t have to wait for all the visuals to load, but it’s not testing what the end user actually sees.
Micro Focus UFT offers different methods for creating a test, from writing the code yourself to a record-and-playback option. Each method requires a different level of coding ability.
Their functional testing and regression test automation is somewhat limited because they only support desktop and web applications (no mobile capabilities) and they only support the Microsoft OS. However, Micro Focus does work with multiple browsers including Safari, Firefox, Chrome, IE, and Edge.
Code-based testing solutions require programming skills for every step of the process. While many of these code-based tools are open source, they’re essentially just a language and library for writing test scripts, rather than an all-in-one test automation framework.
Eventually most teams need a testing grid to run multiple tests at once and test management tools to avoid a bottleneck effect in the testing process.
After discussing four open source tools, we’ll briefly cover BDD and CI tools.
Selenium was originally just a testing framework for automating interactions with browsers. Now, when developers refer to Selenium, it could mean the original app testing framework or the WebDriver.
Here’s an article that goes into more detail about the benefits and challenges of Selenium.
Since Selenium only automates desktop browsers, Appium modifies Selenium to test browsers on mobile devices. Appium can also automate desktop browsers (OSX, Windows, MacOS and Linux), however, they’re best known for their mobile testing platform that supports iOS, Android, and Windows apps.
Appium supports native, web, and hybrid applications and scripting languages using the Selenium WebDriver. Plus, it’s very customizable and can be integrated into any software development tool.
Watir (Web Application Testing In Ruby) is an open source platform built on Ruby (one of the libraries connected to the Selenium WebDriver) but with a more user-friendly user interface than Selenium.
While lines of code in Selenium have to be written very specifically for complex test cases, Watir provides more flexibility.
For example, if your network is slow, you would need to write additional lines of code in Selenium to allow for load time. Watir automatically waits for the page to load. Watir only supports web testing—it doesn’t support mobile apps, API testing, or cross platform testing.
Certain aspects of Cypress are open source (such as their test script editor) but eventually, you’ll need to upgrade to one of their paid tools to run multiple tests or manage your test suite.
Rainforest is a scalable, all-in-one quality assurance solution that’s appropriate for small teams just getting started with an automated testing strategy or QA-mature teams regularly running 500+ automated software tests.
Our free-forever Essentials plan makes software test automation accessible to anyone. This plan has everything you need to get started with automated UI testing without any hidden costs.
You get up to 5 hours of free testing every month and it's only $5/hr after that. Get started for free.
We provide practical guidance on how to start automation testing from scratch and how to choose a test automation tool.
10 automated front-end testing tools that can help you release new features faster with fewer bugs.
With these test automation maintenance tips, prevent automated test breakage and spend less time fixing broken tests.
In this post, you'll learn how to automate regression testing without having to know any code.