Automated software tests are much cheaper to execute, take less time to run, and are less susceptible to human error than manual testing. That’s why most software companies eventually build an automated test suite.
There are many things to consider when first getting started with automated testing, but arguably the most important is choosing the right tool. Choosing the right tool will determine who will be able to write and maintain tests and therefore how quickly and easily you can scale up testing.
Using a no-code automation testing tool that allows anyone to quickly create and edit tests is the most efficient way to scale up testing.
A true, no-code test automation tool enables anyone on the product team to quickly create and edit tests so you won’t have to hire additional personnel for testing.
Our tool, Rainforest QA, was specifically designed to let anyone create automated tests without learning a programming language.
In this tutorial, we’ll provide a few resources to help you build structure and direction for your testing efforts. Then, we’ll provide a step-by-step look at how to start automation testing from scratch using Rainforest QA.
Talk to us about setting up a Rainforest plan that fits your needs.
Knowing When to Start and What Tests to Automate
While automation can help you do more testing in less time and catch more bugs, you need to determine if your application is ready for automation before building any automated tests.
Whether or not your application is ready for automation largely depends on the stability of your product. If your product is still in the early development phase and major code changes are occurring regularly, it’s not time to automate yet—wait until the application under test is more stable.
If, on the other hand, you have a set of user paths that don’t change very often but get tested frequently (smoke tests or regression tests, for example), it’s likely that automation would be a good fit.
In general, the more automated vs. manual tests you have, the more time and money you’ll save—but you can start seeing an impact with just five or ten automated tests. However, some test cases are always a better fit for manual testing, including user paths that:
- Experience constant change (e.g., a promotion page)
- Require human interpretation (e.g., evaluating the clarity of an image or completing a CAPTCHA)
- Have non-deterministic use cases (i.e., if there are multiple, possible correct outcomes from following one path) or complex use cases involving if/then logic.
There are also some types of testing that inherently require human testers:
- Exploratory testing: Where human testers take an unscripted approach to finding bugs that would be missed by all other forms of testing.
- Usability testing: Which tests the overall user experience as well as how the application meets the needs of the user.
Once you’ve determined that your application is ready for automated testing, building a test automation strategy will help you meet your goals quicker and use your resources more efficiently.
You can read more about how to build an automated testing strategy in this guide, but one of the first steps is defining your scope of automation—that is, which test cases, specifically, that you’ll automate.
Deciding Which Test Cases to Automate
Many teams believe that more testing always leads to better quality, which is understandable because more testing can mean fewer bugs get shipped to production. However, more testing is not the goal. If you’re not testing the right things, more testing just means more work—especially when it comes to automated testing.
One of the biggest unexpected costs for QA teams is test maintenance (i.e., the ongoing task of updating tests to reflect changes and new features in your product). Every automated test inevitably needs to be maintained. That’s why any test that isn’t critical for improving test coverage will likely end up costing more to maintain than it’s worth.
To prioritize our testing, we like to use the analogy of a snowplow. Imagine your app is like a city—all the user paths through the app are like the highways, roads, and side streets of a city. When it snows, the snowplow crew follows a predefined route through the city to clear the most trafficked streets first because they affect the most people. Then, if they have the time and resources, side streets with little traffic may get plowed. Likewise, you should start by building tests along the most critical, highly trafficked user paths of your app.
Additionally, you can prioritize which types of testing you’re going to start with. This makes building an automated testing suite more manageable while also ensuring you get the most out of the tests you do have. The two types of testing you should focus on when you’re just starting out are smoke testing and regression testing.
Smoke testing covers the most essential user paths to make sure the application is stable enough to move on to further testing. This is a good place to start because the sooner in the development process you notice a flawed build, the easier it’ll be to fix it.
Regression testing is usually the last step of the testing process before release. It covers all the critical functionalities of an app and deals with ensuring the new build didn’t introduce a bug. Automating your regression suite is especially important for teams following agile software development practices, as we discuss in this article on agile regression testing.
Starting the automation process with even a few smoke or regression tests can save you a significant amount of time and money.
Choosing the Right Test Automation Tool
Finally, the software testing tool you choose plays a critical role in how easily and quickly you can scale up your testing. We recommend looking for an automated testing tool that:
- Lets anyone (even QA beginners) help with QA so you can quickly build a suite of tests without adding headcount.
- Is an all-in-one solution so you can manage all aspects of testing from one platform.
- Doesn’t have hidden fees so you can scale up or down as your needs demand without paying for more than you need.
To learn more best practices for building an automated testing suite, you can read The Rainforest Method here.
Rainforest QA: A True No-code Test Automation Framework
Rainforest QA is a no-code test automation platform for web apps that lets anyone write and maintain tests. Rainforest’s automated tests interact with software via the UI, mimicking the real user experience. This has several advantages over the method used by tools like Selenium WebDriver or Appium (which is basically Selenium but for iOS and Android apps).
Unlike those tools, which identify elements on the page by finding locator IDs in the underlying code, Rainforest uses pixel-matching to identify and interact with elements in the UI. This means that Rainforest tests actually verify both the functionality of the application and the appearance at the same time. It also means Rainforest tests are less likely to break when there are minor code changes that don’t affect the UI.
Here’s how it works.
Write and Edit UI tests without a Single Line of Code
Unlike automated testing tools like Selenium and other open-source tools, Rainforest lets you build an entire test suite without writing a single line of code.
To write or edit a test step, select from a list of preset actions such as click, fill, scroll, etc. Then, use your mouse to take a screenshot of the element you want to apply the action to.
Not only is it easy and straightforward to create any test step, but also anyone looking at a completed test script will be able to understand exactly what’s going to happen. (See the left-hand column in the image below.)
With other user interface test tools, even those that claim to be low-code or codeless tools, you often still need to be able to read and write in JavaScript, Ruby, and sometimes Python to be able to look at a test and know what it’s testing, let alone figure out why it might not be working properly.
To write tests in even less time, you can use embedded tests. Embedded tests let you write a common sequence of steps (a signup process, for example) just once and then insert that sequence into any test that needs it.
Manage Automated and Manual Tests from One Platform
After you’ve built your first suite of tests, you can kick them off whenever you’re ready with one click within the Rainforest platform. Or, you can incorporate your Rainforest tests into your CI/CD pipeline via our API, CLI, CircleCI Orb, or GitHub Action.
With either setup, you have access to our network of virtual machines running different operating systems and browsers. You can run multiple tests simultaneously.
You also have the option to send the test to our community of worldwide QA experts for testing software manually. Many teams find this helpful during the transition from manual to automated testing and for managing manual tests as part of the ongoing testing process.
Understand Test Failures with Video Replays and More
Without the right automation tool, interpreting test results is often one of the most inconvenient, time-consuming parts of automated testing.
Rainforest, however, makes it easy and fast to understand why any test failed. With Rainforest, every test gets recorded and stored on the Rainforest platform, whether it passes or fails. This lets you compare a passed test run to a failed one which can be helpful in understanding failures in flaky tests. Video replays also show you everything leading up to the failure.
Next to every video, you’ll see the test steps with failed test steps highlighted in red. Each test step also has an ‘investigate action’ button which provides further details about how the step was performed.
You can download things like HTTP logs, browser logs, settings, and more, for every test.
If you encounter a real bug, our Jira integration lets you integrate debugging into your workflow by sending a ticket to the software development team from right within Rainforest.
The ticket automatically includes steps from the failed test, a screenshot from the failed test step, a link to the video replay, and more information in Rainforest.
All-in-One Solution
Many automated testing tools will only offer a way to write and run individual tests, but most teams find they eventually need a way to manage dozens or even hundreds of tests, run multiple tests at once, and communicate more efficiently with everyone on the team.
Rainforest is equipped with everything you’ll need to run and manage a full suite of automated and manual tests, including:
- Test suite management tool for keeping a large number of tests organized.
- Access to multiple operating systems and browsers, including current and older versions of Safari, Chrome, Firefox, Internet Explorer, etc.
- Built-in test data like randomized email addresses, credit card numbers, names, etc.
- Integrations with Slack, Microsoft Teams, and email so your team can get real-time testing reports for test failures, run completions, and more.
- Unlimited team members.
Talk to us about setting up a Rainforest plan that fits your needs.
