Many QA teams choose Cypress.io for creating automated software tests because it executes tests directly inside the browser (which solves some of the most frustrating timing issues that are common with Selenium).
However, many teams end up looking for Cypress alternatives because it still has a few major downsides:
- Cypress tests can be time-consuming to set up. Many teams find they need to add custom attributes to element locators because the underlying code of modern web applications are often complex.
- Cypress tests are limited to performing actions inside a single browser, which means critical functions of some applications will go untested. To test functions like downloading an application to a desktop, installing browser extensions, or testing interactions between users (such as chat features), you’ll need your testing tool to be able to test outside a single browser window.
In this post, we’ll discuss these downsides in more detail and show how our tool—Rainforest QA—solves these issues. Then, we’ll provide four other tools for you to consider.
Table of Contents
- Rainforest QA
Talk to us about setting up a Rainforest plan that fits your needs.
Rainforest QA is a codeless functional testing tool that uses a drag-and-drop editor to create tests that interact with the visual layer of your app the way a real user would.
Here are five reasons why Rainforest QA is the best Cypress alternative for many teams:
1. Anyone Can Write and Maintain Tests without Adding Custom Attributes
Cypress tests look for element locators in the underlying code of an application to perform actions and make assertions. However, modern front end website and application building tools usually generate complex code which can make it difficult to quickly identify where each element locator begins and ends. To solve this problem, many software development teams add custom attributes to each element to make them easier to find. This is fairly easy to do as you build a new feature, but adding custom attributes to existing features can be quite labor intensive.
Rainforest QA lets anyone write tests by testing the visual layer directly rather than the underlying code. To test the visual layer, Rainforest tests use pixel-matching to locate and interact with elements on the UI. You’ll never need to worry about locators or adding custom attributes.
To write (or edit) test steps in Rainforest QA, simply choose from a dropdown menu that lists possible actions, such as ‘click’, ‘fill’, or ‘verify’. Then, take a screenshot of the element you want to apply the action to by clicking with your mouse and dragging a box over the element.
Once you write a test, you can easily embed it in other tests to speed up test writing and maintenance. We call these “embedded tests.”
Embedded tests allow you to write a common sequence of test steps (e.g., for logging in or adding an item to a basket) and insert those steps into any test that needs it. Then, you can edit a test step in one place and it will automatically be updated in every other test it’s embedded in.
When you’re ready to run your Rainforest tests, you can manually kick them off within the Rainforest platform. Or, your dev team can start a run from within your CI/CD pipeline using our CircleCI Orb, GitHub Action, API, or CLI.
You’ll have the option to choose to run tests on current and older versions of any of the four major browsers: Safari, Microsoft Edge, Chrome, and Firefox.
Finally, because Rainforest QA lets anyone create and edit tests, you’ll be able to free up developers’ time so they can focus on building new features and keeping the software up to date.
2. Tests Can Include Actions Outside of a Browser Tab or Browser Window
Like most test automation tools, Cypress tests are limited to testing actions within a single browser tab. For many teams, this means critical functions of their software will go untested (or will have to be tested manually). For example, a few common regression test cases that require actions that leave the browser are:
- Testing interactions between two users: There are many applications with chat functions where it’s useful to open two separate browsers to actually test the sending and receiving of messages between two users.
- Downloading a desktop app: If users have to download your app to actually use it, it’s extremely important to verify that this functionality is actually working. To test this, you need to be able to leave the browser and interact with the operating system of the device to verify a download and ensure the app can be opened after the download.
- Installing a browser extension: If your entire product relies on users installing a browser extension, you’ll want to test that process, which requires the ability to interact with the browser settings—something you can’t do with Cypress.
With Rainforest’s proprietary automation, you can create tests that open new tabs and new windows, open and close browsers, and even interact with desktop apps.
If you want to see it in action, check out this video walkthrough of how to create a test that saves an online image file to the desktop using a keyboard shortcut, then uploads the file to Google Drive.
3. Tests are Less Brittle to Minor Changes in the Underlying Code
As mentioned above, Cypress tests search for element locators in the underlying code of a web app to execute test steps. If the element locator is present, the test will pass. However, if there’s even a slight change in how the locator was named (it’s missing a letter, for example), the test will fail—even if there’s no change to the UI.
There are also scenarios where a Cypress test would pass even though there is a bug visible in the UI. For example, the element locator could be present but a pop-up modal might be covering the corresponding UI element. Or, a bug in the CSS that defines the positioning of an element could cause it to render off-screen.
A Rainforest test will only fail if there’s a change to the UI. This means Rainforest tests are immune to minor changes in the underlying code that don’t affect the UI. And, Rainforest tests are much more reliable for catching bugs on the visual layer.
Sometimes you may want to make deliberate changes to your visual design without having to update your tests right away. For these situations, Rainforest has a text matching feature.
Text matching verifies the text content of the element rather than other visual aspects of the element. For example, both buttons below have the same text but their appearances are very different.
With text matching turned on, either button would result in a passing test in Rainforest. Without text matching, the test will pass only if the current version of the button matches the original screenshot.
4. Investigating Test Failures with Video Replays Is Fast and Easy
Figuring out why a test failed is an ongoing task that can be very time-consuming without additional features or tools to assist you. With Cypress, you’ll have two main options that can make it easier to identify why a test failed:
- You can run your tests using Cypress’s API which will automatically take snapshots of the underlying code at the point of failure and will include a message with additional details.
- You can run tests from Cypress’s Command Line Interface (CLI) which allows you to record videos of the lines of code as the test is being run (but you’ll have to set this up manually for every test run and the videos will automatically be deleted during the next test run).
Rainforest QA, on the other hand, automatically records every test run, whether it passes or fails, and saves the recording for future reference. Oftentimes, comparing a successful test run to a failed test run is the key to figuring out the cause of the failure.
Being able to see everything that happened leading up to the failure can also be very useful.
Video replays will show you exactly how the failure would’ve appeared to a real user so you can decide how important it is to fix the issue.
Rainforest QA also provides HTTP logs and browser logs, which can point to whether or not the test failure was due to hiccups in the test environment.
Finally, Rainforest QA comes with a Jira integration so any tester assisting with the testing process can automatically create a ticket to send to the development team for debugging. The ticket will automatically include the failed test steps, a screenshot of the failed test step, HTTP logs, and a link to the full test results and video recording in Rainforest.
5. Only Pay for What You Use
This plan has everything you need for test automation, including parallel testing on virtual machines, test management features, Slack and Microsoft integrations, unlimited team members, and much more—without any hidden costs or signed contracts.
The Startup plan offers usage-based pricing. You’re charged for how much time you spend running tests, and that’s it. There’s no charge for the time spent previewing, writing, editing, or otherwise managing your test suite.
By comparison, the free version of Cypress allows for multiple tests to be run at once, API integration, analytical reporting, and community support. But you only get access for 3 users and up to 500 test results per month.
If you want flake detection features, Jira integrations, the ability to cancel runs after a failure, direct support (meaning support other than online guides and community support), or more users and more tests, you’ll have to choose a monthly plan. The monthly plans are priced at $75/month or $300/month based on features, number of users, and how many test results you need. You can also contact the Cypress team for a custom plan.
Talk to us about how Rainforest QA can help you ramp up testing without adding headcount.
Katalon is built on Selenium and Appium and was designed to be used by both developers and non-developers for web testing and mobile app testing.
They offer a record-and-playback feature which lets non-developers create test scripts. Then, unlike some record-and-playback tools, Katalon lets developers stop and restart a recording at any point during test creation or execution to insert additional lines of code. This makes it much easier for developers to create complex test cases and fix broken tests.
Katalon has a free version of their software, but it’s only intended for individual use on simple platforms. The free version lets you run a test and know whether it failed or not, but you won’t have access to screenshots, recordings, cross-platform support, or suggested fix features.
Testim.io was built to be used with other tools such as TestRail, GitHub, Applitools, BrowserStack, and Sauce Labs. However, it doesn’t support iPhone, Android, or iPad platforms. Its paid plans only allow you to run between two and eight concurrent tests, and there’s an absolute limit to the number of tests you can run in a month.
You can configure WebdriverIO to run on the WebDriver Protocol (for cross-browser testing) or the Chrome DevTool Protocol (for automation using Puppeteer). WebdriverIO also supports native mobile application testing for both Android and iOS.
Finally, WebdriverIO offers many different third-party plugins so you can create a test automation framework that meets all your needs.
Protractor also supports behavior driven development with integrations to Mocha and Jasmine.
Make the Switch to Rainforest QA
With Rainforest QA, anyone can write and maintain automated UI tests without learning a new programming language.
Talk to us about setting up a Rainforest plan that fits your needs.