The goal of codeless (or no-code) test automation tools is to make the creation and maintenance of automated software tests accessible to anyone connected to the product. This may include product managers, devops, CSMs, QA specialists, developers, or anyone else.
Many software teams are choosing to go codeless because it’s the only way to keep up with new releases in a continuous integration/continuous delivery pipeline and achieve adequate test coverage.
But choosing the right codeless automation testing tool is important. Dozens of tools call themselves “no-code test automation tools” or “codeless Selenium tools” and claim that anyone can become an automated testing expert with them, but they aren’t all the same.
Some tools are best suited for testing marketing websites because they’ll immediately catch any visual inconsistencies after a software update. However, they create huge maintenance headaches when used with a complex web app.
Other tools are best for huge enterprise software teams but may be overkill for most web application SaaS products. One theme of many of the tools is that even if you don’t need to have coding skills or know how to write Selenium scripts to create the tests, you’ll eventually need to know how to do both to create sophisticated tests or maintain your test suite.
Here’s what to look for in a codeless automation tool:
- The ability to run your most time-consuming and repetitive tests, which are typically your smoke tests and regression tests, without knowing any programming languages.
- A user interface that makes it easy to maintain your test suite as your product changes.
- The ability to test on multiple operating systems and browsers.
- A pricing structure that makes sense for the size and maturity of your company.
- An all-in-one solution that allows you to run tests with a click of a button.
In this article, we’ll analyze the features of 10 codeless test automation tools, starting with our tool, Rainforest QA.
Rainforest QA: Improve Quality Without Adding Headcount or Learning Selenium
With Rainforest QA, you can perform end-to-end UI testing without writing a single line of code. Rainforest is great for software teams who are looking to improve the quality of their web apps without adding headcount.
Specifically, as we explain in this article, Rainforest allows non-developer team members who have a stake in product quality to help with (or even outright “own”) running and maintaining tests.
Some of the features Rainforest users love the most include:
An All-In-One Solution for Automated and Manual Software Testing
Rainforest QA offers both a beginner-friendly platform for creating automated tests without code along with access to human testers for the tests that don’t make sense to automate.
Most of the other tools available offer either a codeless platform for writing automated tests, or a crowdsourcing platform for outsourcing testing to humans. Rainforest combines both seamlessly.
The automated testing platform allows users to create, run, and manage any number of automated tests all at once without additional software or infrastructure like a testing grid. And the crowdsourced testing platform helps teams offload manual testing for uncovering edge cases or exploratory testing.
Watch this 2-minute video walk-through of a test for Airbnb’s search function to see the platform in action.
Test What the User Sees Instead of the Underlying Code for Higher Confidence in Your QA Process
Rainforest tests in the same way humans interact with an app. The automation interacts with the visual layer of the app with mouse clicks, keyboard actions, and visual confirmations. This is a major distinguishing feature of Rainforest compared to all of the other major codeless testing tools, which are built on top of the open source automation framework Selenium. Those tools test the underlying code that renders the visual layer, but they don’t actually test the visual layer, which means they’re not actually testing what a user will see.
This can affect the accuracy of your tests. When you only test the underlying code, you can end up with tests that pass because all the elements the test is looking for exist in the code, but they could be hidden or completely unusable on the visual layer.
For example, the “Add to Cart” button on an e-commerce site could be present, but it might be blocked by a pop-up modal the user can’t remove. That’s an example of a test case that would pass in most tools that test the underlying code, but that test would fail in Rainforest—alerting the QA team about a high-priority bug.
An added benefit of this approach is that, since Rainforest doesn’t generate Selenium code (or any code at all), no one on your team needs to know Selenium in order to edit, maintain, or debug tests.
Test Actions Outside the Browser
Most codeless tools generate scripts for automating actions inside of a single browser tab. If you want to test user experiences that involve opening and closing browsers or browser tabs, you can’t. But because of Rainforest’s proprietary automation design, you can create tests that open new tabs, open and close browsers, and even open desktop apps.
These features help test how your app interacts with desktop apps, how it interacts with browsers with ad-blockers, and other complex functions.
For example, here’s a video walkthrough of creating a test that saves a file to the desktop from one location, then uploads the file to Google Drive.
A Clean Virtual Machine for Each Test
When you run a test in Rainforest QA, the automation creates a brand new virtual machine (VM) in the cloud to run your test.
Using a new VM for each test gives more consistent, repeatable results than testing on real devices (which is what many other codeless testing tools do) because it eliminates a potential source of variation between test states.
If you test on real devices, you introduce the potential for inconsistent test results because you don’t know what settings, defaults, or cookies are on that device.
Maybe the device on one test operating system has anti-virus software, another browser has ad-blockers, and another has unique privacy settings. Any of these features could introduce bugs that would be difficult to reproduce and identify, which just creates headaches for your QA team.
Tests That Are Easy to Maintain
All automated software tests, codeless or coded, need to be maintained. No matter what tool you use to create and run automated tests, you’ll inevitably get tests that break because the application or website changed.
Codeless tools that test the underlying code (again, that’s all of the tools except Rainforest) use element locators—snippets of code that identify each element—to find the objects to apply actions to. If the test can’t find that locator, it’ll fail. This causes tests to become flaky—meaning they frequently return false or meaningless failures—because developers often change locators of existing features when they’re fixing bugs or adding new features. The first generation of codeless testing tools were notorious for often breaking because of this.
To solve this, most codeless testing tools add alternative or backup locators that are used if the first one can’t be found. These alternative locators make the tests much more stable than they used to be, but they’re not foolproof.
Rainforest does away with these code-based locators completely and instead uses pixel matching and text matching to find elements on the screen. Rainforest tests can still break when the visuals of the page change, but those failures are usually meaningful to testing teams because they can affect the user experience.
For example, let’s say you work for Notion and want to test the signup flow for your app, which starts with clicking the “Get Notion free” button. When creating the test, you’d identify the element you want to test—the button—by clicking and dragging a selection box around the button in a live preview of app within Rainforest. Instead of a code-based locator, Rainforest will use this small screenshot to find the element during the test. If those pixels exist anywhere on the page, the test will find them and be able proceed, even if the underlying code changed.
Rainforest QA tests aren’t totally immune from “false positives” (test failures for which there’s no actual bug) in that a minor UI change, such as a button slightly changing shape, may break a test. But Rainforest has several features that help to minimize maintenance during the testing process.
When a test fails, the software suggests fixes automatically, and it allows you to quickly edit individual steps of the test. Most users find they can edit tests in a few seconds with these features.
With other codeless tools that use a web recorder to create tests, you often have to completely re-record the test to fix errors. Most of them have a self-healing feature that uses AI to try to replace missing locators, but if that doesn’t work, you’ll either have to manually edit the Selenium code or recreate the whole test from scratch.
Rainforest also offers a test-writing service for an added fee. Our team of skilled crowd-testers can create new tests for you, or they can go through your regression testing suite and fix broken tests after a major software update.
It’s a great solution for small QA teams who don’t have enough people to keep up with test maintenance and need to speed up their test creation.
Video Replays of Every Test
Rainforest QA records a video of every test, whether it passes or fails. You can review those videos after each test execution round to quickly identify why a test failed.
If it failed because of a legitimate bug, you can easily identify the bug and send a ticket to your developers. And if it failed because the test is broken, it’s easy to identify what needs to be fixed in the test.
The video recordings also capture more than just what a human would see during the test. We record data about the browser settings, the network traffic, and a variety of other factors that could affect the software performance. This data can help developers fix bugs faster.
Test on the Big Four Browsers
Most QA teams will want to be able to run tests in the “Big Four” browsers: Safari, Microsoft Edge, Chrome, and Firefox. Rainforest handles all of those seamlessly, and because Rainforest tests the visual layer, it’ll be easy to add support for browsers and operating systems that get invented in the future.
Built-in Tools for Testing Complex Signup Flows with Email Confirmation
Rainforest has a built-in library of test data that makes testing signup and login flows easy, even if they have a confirmation step or two-factor authentication. Rainforest can generate virtual inboxes, plus random email addresses, passwords, SMS numbers and inboxes, SSNs, and more.
Watch this 4-minute video to see how easy it is to use the email/inbox generator in a sign-up test.
Virtual Credit Card Testing
Rainforest offers an add-on feature, for an additional fee, that allows virtual credit card testing to test payments processing on your production site without using gift cards or corporate credit cards.
Most QA teams just test e-commerce sites with dummy credit cards in a staging environment. This process catches most bugs, but if the production site starts experiencing problems, members of the product team often end up testing the production site with their personal credit cards, which creates a big hassle.
Accessible Pricing Structure
Rainforest offers a free 14-day trial. You can sign up and see how it fits into your software development process all without talking to a sales rep.
And our pricing structure is the same whether you’re a one-person QA team or an enterprise software company. You only pay for what you use, so if you just want to create a regression test suite of 20 tests to start out, you’ll only pay for running those 20 tests.
Now we’ll take a look at some other codeless test automation solutions.
Testim.io
Testim.io is a tool that uses a Chrome extension to record test steps for basic tests. To create more complex tests, it requires a user to insert custom code and offers a built-in JavaScript editor.
It allows users to reuse custom code across different tests, which can lower maintenance. It has integrations with TestRail, GitHub, Applitools, BrowserStack, and Sauce Labs.
However, Testim.io doesn’t support iPhone, Android, or iPad platforms. Its paid plans only allow you to run between two and eight concurrent tests, and there’s an absolute limit to the number of tests you can run in a month.
Rainforest allows you to run any number of tests in parallel and has no cap on the number of tests you can run in a month—you simply pay for what you use.
SmartBear TestComplete
TestComplete is a great tool for enterprise software companies that need to test desktop apps. It has a ton of features, but most SaaS companies building web apps will find it overwhelming.
TestComplete offers keyword-driven and data-driven testing, cross-browser testing, API testing, and CI integrations.
Mabl
Mabl is a record-playback codeless test tool. You can record tests with a Chrome extension, but then you have the option to run them in Firefox, Internet Explorer, and Safari.
It has a very informative results interface, showing high-level statuses on passed and failing tests, JavaScript errors, broken links, and visual changes.
You also can store common test sequences as “reusable flows” and insert them into future tests, which saves time when creating tests.
One downside is that Mabl doesn’t offer any kind of crowdsourced testing to help teams scale up their manual testing.
Applitools
Applitools uses screenshots to identify visual changes between different software releases. It’s one of the best tools available for testing marketing sites, where it’s important to verify that the visuals stay exactly the same (brand colors, logos, fonts, etc.).
For more complex web apps, this style of visual testing isn’t ideal because it’s very sensitive to visual changes that don’t affect the user experience.
Screenster
Screenster is another record-playback, UI-based testing automation platform.
Like Applitools, it’s good at catching any visual differences between test runs, but that means it can return failure results even when the app is functioning perfectly.
Ghost Inspector
With Ghost Inspector, you can create codeless Selenium tests with a record-and-playback browser extension and edit them with a codeless editor.
However, some users say the codeless editor is not user-friendly for testers without a programming background.
Ghost Inspector offers a nice tool for automated website monitoring. With it, you can schedule tests on your production site to check the functionality regularly and alert you if anything breaks.
It currently only works with Chrome and Firefox, but more browsers are on the way.
Another drawback of Ghost Inspector is that you often need to use JavaScript to add custom code to make tests less flaky.
Katalon Studio
Katalon Studio is a test automation platform built on top of the Selenium and Appium frameworks.
It has an easy-to-use interface that helps testers without a programming background create tests with a drag-and-drop manual editor.
It also offers tools for more technical users to add custom code with Selenium or JavaScript for advanced test automation scripts.
Katalon Studio has a number of integrations that help agile teams to better collaborate (Slack, GitHub, Microsoft Teams, and more).
Perfecto
Perfecto, formerly TestCraft, is a codeless testing solution with a built-in testing grid that uses artificial intelligence and machine learning to automatically generate Selenium test scripts.
Preflight
Preflight allows you to create automated tests with a web recorder and playback tool. It offers built-in libraries of randomized data for completing forms.
PreFlight automatically generates a new email address with an inbox for each test, allowing for seamless testing of sign-up flows.
It supports Chrome, Firefox, Edge and Internet Explorer.
Preflight is still an early product, but it has some promising features that could make test maintenance less of a burden.
It notes that the web recorder uses more than just element locators to identify objects in tests (which could make tests more stable), but it’s unclear what else it uses.
Start Codeless Automation Testing in Minutes with Rainforest QA
It’s easy to get started with automation testing with Rainforest QA’s codeless solution.
Non-technical users find the software easy to use with a short learning curve. It handles a wide variety of complex test cases, allowing teams to automate more tests and gain more confidence in their QA process.
Whether you’re just dipping your toes into QA or you’re a mature team with lots of test automation experience, Rainforest QA helps you build better apps faster. Try it yourself by starting a free 14 day trial.
