Selenium is the oldest and most well-recognized automated testing tool for web apps, so a lot of software teams try it out when they first experiment with automated testing. But most teams quickly run into three Selenium disadvantages:
Because of these challenges, many software companies that use Selenium find testing to be a huge drain on time, effort, and resources. That’s why we designed Rainforest QA—to solve the limitations of Selenium and make QA easy to scale and maintain.
In this post, we’ll discuss each of the disadvantages of Selenium in detail and show how Rainforest QA solves those problems.
Sign up for Rainforest QA to save time and money on test automation—you can run five hours of no-code automated tests for free, every month. It’s only $5/hour after that.
Selenium tests search for element locators in the underlying code of an application in order to interact with web elements to perform test steps. Usually, these steps involve interacting with the element to move the test forward (e.g., clicking the submit button) or verifying the presence of an element. You can also add test steps that verify various visual settings of the element, such as size, position, or color. If the element locator is present and the settings are correct, the test should pass.
However, this approach to test automation has two inherent downsides:
While there are workarounds to help ensure you catch visual bugs (more on this in a later section), in the end, Selenium tests make assumptions about elements on the UI rather than testing what real users will actually see.
Rainforest QA tests interact with the visual layer of your application. Instead of searching for element locators in the underlying code of an application, Rainforest tests search for elements (like buttons or form fields) based on a matching combination of pixels in the UI.
We provide a detailed tutorial about how to write a Rainforest test in a later section.
This is the closest to how a real human would interact with the application because the test only uses what’s available visually and doesn’t touch the underlying code. This also means Rainforest tests won’t break because of minor changes to element locators that don’t affect the UI.
Even though Selenium supports many programming languages, developers using Selenium usually find that writing and maintaining test scripts is very time-consuming. Not only do they have to learn a new testing framework, but each individual test script requires a lot of set up and coding. To write Selenium test scripts:
There’s a Selenium WebDriver that supports multiple programming languages (including Java, Python, Ruby, C#, JavaScript, Perl, and PHP) and a record-and-playback tool (called Selenium IDE) which can make it easier to write test scripts. In the end, however, you’ll still end up writing or generating test scripts built on code. Whenever a test fails or needs to be updated, you’ll still have a ton of code to sort through.
Rainforest QA is a true no-code platform that lets anyone create (and maintain) tests in a matter of minutes whether they have a technical background or not.
To create (or maintain) a test step in Rainforest, you choose from a dropdown menu of preset actions—such as ‘click’, ‘fill’, or ‘scroll. Then, you take a screenshot of the element you want to apply the action to by clicking and dragging the mouse over the element.
To verify the visibility, color, shape, etc. of an element, you only need the one screenshot.
Because Rainforest tests interact with the UI directly, you won’t have to add any custom attributes to your application before writing tests.
Additionally, Rainforest QA has built-in features to help make tests more resilient to changes in the application. Not only are Rainforest tests immune to minor changes in the underlying code of the application that don’t affect the UI, but you can also choose to use text matching for any test step. With text matching turned on, Rainforest will search for the text content of an element rather than just the appearance of the element.
For example, both of the buttons below say “Buy Now” even though their appearances (i.e., color and shape) are different.
If text matching is enabled, the test will pass with either version of the button. If text matching is not enabled, it will only pass if the original screenshot matches the current version being tested. This gives you the flexibility to decide for each test step whether you want to test the entire design of the element or just the content.
After a test run, you’ll want to see which tests passed and which ones failed. Selenium doesn’t have any reporting capabilities for test results. Instead, you’ll have to click into each test individually to see if it passed or failed, or add on other test management tools, such as TestNG, JUnit, or Maven.
Once you know which tests have failed, you’ll need to figure out why they failed. Selenium doesn’t offer any built-in features to help you identify why a test failed, which means it could take hours to find the cause of the failure.
You can add additional code so that a snapshot of the underlying code is automatically taken at the point of failure. However, in many situations, tests fail because of a bug in a previous part of the test. In these situations, one snapshot at the point of break won’t be enough to tell you why the failure occurred.
For example, let’s say you want to verify that a ‘Download was successful’ message appears at the end of your test. A snapshot of the underlying code could tell you that the message didn’t appear but it can’t tell you if that’s because the download itself was unsuccessful or if an error with the confirmation message is the issue.
With Selenium, you’ll most likely end up sorting through tons of code to find the small errors that caused the failure. This is often too slow and labor intensive for teams that need to release quickly and frequently.
Rainforest QA automatically creates test reports for each test run that clearly show you which tests passed, which ones failed, and the run history for each test.
Rainforest also automatically records a video of every test whether it passes or fails. This lets you view the failed test step and every step leading up to it. You can also compare a failed test to a successful test, which can be really helpful when the reason for a failure isn’t immediately obvious.
Failed test steps get highlighted in red along with a brief explanation of why the step failed and an ‘investigate action’ button.
The ‘investigate action’ button lets you see more detail about the failure including the closest match found for both the appearance of the original screenshot and the text content (if text matching was enabled).
Then, if there’s a real bug, anyone can automatically send a ticket to Jira for the development team to fix the bug. The ticket will include the failed test step, a screenshot of the failed test step, HTTP logs, and a link to the full test results and video recording in Rainforest.
Selenium automates web browsers, which means each test is limited to testing within a single browser tab. For many teams, this means they won’t be able to create test cases for some of the critical functions of their app. For example:
Rather than just automating browsers, Rainforest QA automates entire virtual machines running operating systems including Windows, MacOS, iOS, and Android. Rainforest tests can interact with anything you see on a screen, which means you’ll be able to perform any of the operations described above. Here’s an example video of how to create a test that saves a file to the desktop from one location then uploads the file to Google Drive.
Automation is the fastest, most effective way to run software testing in most situations. But there are some tests that are better suited for manual testing. A test is better suited for manual testing if any of the steps require subjective interpretation or are subject to frequent change (as is the case with many test cases that cover brand-new features).
Most teams using Rainforest QA find that automated testing can be handled so efficiently that they have no problem handling a few manual tests in-house. But, Rainforest QA also offers the fastest manual testing available via our community of QA experts.
Any test written in the Rainforest visual editor can also be sent to the Rainforest community for manual testing, where results will be returned in 17 minutes on average.
While the Selenium automation tool is open-source, it only provides a way for you to write and run tests on your own devices. If you want to run multiple tests simultaneously, manage a large number of tests, or simplify communication with the QA team, you’ll eventually need to add additional (paid) tools and services.
Rainforest QA is equipped with everything you’ll need to run and manage a full suite of automated and manual tests, including:
Our free-forever Professional plan makes software test automation accessible to anyone. You get 5 hours of free testing every month. It’s only $5/hr after that and you only pay for what you use. That is, you only pay for test execution, not for previewing, writing, editing, or otherwise managing your test suite.
The Professional plan includes a 14-day trial of manual testing by our human testers. After that, manual testing is just $25/hour.
We provide practical guidance on how to start automation testing from scratch and how to choose a test automation tool.
10 automated front-end testing tools that can help you release new features faster with fewer bugs.
With these test automation maintenance tips, prevent automated test breakage and spend less time fixing broken tests.
In this post, you'll learn how to automate regression testing without having to know any code.