When a software company realizes its testing needs have outgrown its in-house quality assurance team’s ability to keep up, the first thought is often to hire an outsourced quality assurance provider—such as Applause—to run the company’s entire testing program.
While this option for increasing test coverage can help you avoid adding headcount, it has a few major downsides:
- It’s almost impossible to fully communicate the necessary product-specific knowledge and business goals to an outside team to make sure they’re testing the right things and identifying bugs that actually have an impact on the business.
- External teams often have a hard time adapting to rapidly changing test coverage priorities between release speed and test coverage.
- The time between requesting a test run and getting the results back is often too slow for teams who frequently release new software updates.
When your test coverage needs are constantly changing and growing and you need test results as quickly as possible, there’s really no substitute for managing QA internally. But that doesn’t mean you have to hire dozens of QA specialists.
We developed Rainforest QA as a super fast testing solution that keeps your team in control of QA while relieving them of the burden of tedious, time-consuming testing.
In this post, we’ll show you how Rainforest QA can help you scale testing without adding headcount or slowing down your release cycles. We’ll also cover eight other Applause competitors for you to consider.
Table of Contents
- Rainforest QA
- Test IO
- Mindful QA
Want to scale up your QA testing without adding headcount? Talk to us about a Unlimited plan featuring 24/7, on-demand access to manual testers from our worldwide community of QA specialists.
Rainforest QA: The Speed of Automation and the Flexibility of Humans in One Testing Platform
Services like Applause were created before the idea of continuous release became the norm in software development. These services are more like traditional outsourcers, meaning a team outside your company takes full responsibility for some or all of your test coverage. This approach can work for well-established companies using legacy code that rarely needs to be updated.
But if you’re a modern software startup and you’re releasing new features and updates multiple times a week (or even multiple times a day), you’ll need a faster and more responsive QA solution.
With Rainforest, your team stays in control of your QA strategy and test plan, but you get on-demand access to a team of testing experts who are available 24/7 to physically run the tests. On average, our testers return results for test suites in less than 20 minutes since we process all tests in parallel.
Additionally, our all-in-one proprietary no-code automation platform lets anyone easily manage a fully-automated test suite.
Here’s how it works:
Get Manual Test Results in Less Than 20 Minutes
Most outsourcing services require a long onboarding process so their team can become familiar with your goals and process, develop a QA strategy for you, and then create your tests. This process can take somewhere between hours and weeks to complete.
With Rainforest, you can skip the long onboarding process and start creating your tests right away. No matter how many tests you submit at a time, QA specialists from our global community will start running your tests right away, and all your tests will get executed simultaneously (i.e., in parallel). This allows us to return manual test results in an average of just 17 minutes after submission.
While many manual testing services claim you get results within minutes, this usually means within minutes after the tester begins the test—not within minutes of you submitting the test.
At Rainforest, our QA specialists are available 24/7, even on holidays, so you get the fastest results possible right when you need them.
Everyone in our tester community has been working with us for at least five years, undergoes ongoing training, and carries a constantly-adjusted reputation score. Beyond these measures for consistently delivering high-quality test results, we also send every test to at least two testers. These testers have to agree on the results, otherwise the test gets sent to additional testers for confirmation. But you’ll only be charged for the time it takes one tester to complete the test—regardless of how many testers it takes.
Finally, as our testers execute your test scripts, they’ll note any unexpected or undesirable aspects of the application they encounter. These might not be bugs, per se, but could be things affecting the user experience that you’ll want to address. For example, if the tester has a hard time finding a button because it was too small or the page was too visually distracting, they’ll make a note of that.
You have two options for writing manual tests: the Visual Editor or the Plain-Text Editor. We’ll describe both in detail to show you how Rainforest helps to mitigate the potential communication errors that can happen any time you’re working with human testers.
The Plain-Text Editor provides a structure for writing out test instructions in plain English.
Each step will have two parts: A tester instruction and tester confirmation. Each tester confirmation should always consist of a simple yes or no question to validate the element being tested.
We provide in-depth guidance and tips for writing tester instructions. These best practices help mitigate any miscommunications about what should be tested and the desired results.
If any of the steps are unclear, our testers will ask you to clarify your intentions rather than just make a guess. These policies allow you to have total confidence in the accuracy of the test results.
For each test step, you choose an action (e.g., click, fill, observe, etc.), take a screenshot of the element you want to apply the action to, and/or choose how to apply the action (i.e. ‘fill’ ‘screenshot’ with ‘random email’ or ‘wait’ for ‘3 seconds’).
For any tests created in the Visual Editor, you can click a button to export the test steps into plain English instructions for the manual testers to follow.
Unlike the Plain-Text Editor, the Visual Editor lets you include screenshots of the exact element you want the tester to interact with. This can be very helpful for mitigating potential miscommunications that can happen when testing complex applications.
When you create tests with the Visual Editor, you can run them using the Rainforest community of testers (as we showed above) or with our automation system, so you’ll have the flexibility to choose either method at any given time.
Write and Update Automated Tests In-house or Utilize Rainforest QA Specialists
While most QA teams will always have a few manual test cases that aren’t a good fit for automation (because they require human interpretation, are too complex, or cover evolving features), most teams will eventually want the majority of their testing to be done using automation since it’s faster and cheaper.
(You can read this article to learn more about when to use automated testing vs. manual testing.)
If you’re using the Visual Editor to write test scripts for the Rainforest community of testers, it’s easy to make the switch to automation since the same test scripts can be used for automation or manual testing.
If you’re using the Plain-Text Editor for manual testing, it’s still quite easy to switch to automated testing because anyone—including non-technical team members—can create, edit, and run Rainforest tests.
Here’s a video that walks you through creating a Rainforest test in more detail or you can refer back to the Visual Editor section above.
With Applause (and some other testing services), the way they handle automated testing is to assign a team of their software engineers to create and edit automated tests for you, typically using a testing framework like Selenium. With this type of service, you rely entirely on their team to keep tests updated—you won’t have the option to do it yourself.
While this can be useful in some situations where a large amount of tests need to be updated at the same time, many teams find that relying on a team outside their company to keep tests updated isn’t practical for small, frequent updates. For every update to the application, your team will have to convey the details of the update to the outside team and wait for changes to be made before running the test again.
Rainforest QA gives you the flexibility to choose whether to update your automated tests in-house or get help from our team of QA experts.
Using the no-code Visual Editor, anyone on your team can easily and quickly create or update tests.
If you need additional help writing or updating your tests, you can use the Rainforest Test Writing Service, another service available in our Unlimited plan. You simply record a video or write a document describing what needs to be tested and how, and our QA experts will create (or edit) passing tests that are optimized for the Rainforest platform.
Choose What Testing is Important for Your Team
A lot of manual testing services will design your QA strategy and test plan for you. Then, they’ll write the test scripts and decide which tests need to be run based on whatever agreement you made with them. For example, the testing company might agree to complete 80 hours of testing per month or to run your regression suite four times per month. In either scenario, you’ll have to sign a contract that agrees to a consistent amount of testing—even though your needs may change from day to day.
Outsourcing your testing also means you’re trusting that someone outside your team will be able to understand your product, goals, and processes well enough to provide valuable testing results. Additionally, you’ll have little say on a day-to-day basis as to which tests need to be prioritized and which tests could be skipped in order to meet release deadlines.
We’ve found teams are able to achieve better test coverage while meeting more deadlines if they’re able to internally manage the tradeoff between test coverage and speed at any given moment. You can access Rainforest’s crowdsourced manual testing on an as-needed basis so you have full flexibility when it comes to choosing which tests to run and how often to run them.
We’ve also found that teams are better able to achieve their quality assurance goals if they design their QA strategy and test plan internally. (We go into more detail about how to do that in this article on QA test strategy and this article on how to build a test plan.)
Of course, if you’ve never developed a QA strategy before, it pays to have expert guidance. As part of our Unlimited plan, you’ll get a dedicated customer success manager to help you develop a testing strategy that makes sense for you.
Get Reliable Results Every Time
All Rainforest tests—including any testing done by the Rainforest community of testers—are run on our network of cloud-based virtual machines. This is particularly important for manual testing because most manual testing service providers offer ‘in the wild’ testing, meaning the manual testers are using their own devices to execute tests. This may sound useful at first—‘real users with real devices’—but this approach can affect the reliability of your software testing.
Functional software testing is kind of like culturing bacteria in a laboratory. Your sample has to be placed in a perfectly clean environment for you to be confident that any bacteria that grows was present in the sample and wasn’t introduced by outside factors. To prove those results through repetition, the same contamination-free environment has to be used every time.
In the same way, software testing needs to be done in the same clean environment every time to get consistent, reliable results and to make it easier to reproduce bugs.
Rainforest virtual machines provide this clean environment so all your testing is free from unpredictable outside factors, such as ad blockers, browser security settings, or outdated operating systems.
Note: Rainforest QA does not support hardware feature testing (GPS, recording and playing audio and video, etc.). However, our customers have found they can easily handle these test cases in-house with the time they save using Rainforest for their functional testing.
Quickly Understand Every Test Failure with Video Replays and Detailed Logs
Every Rainforest test gets recorded and can be played back at any time. This is true whether the test passes or fails and whether it was run using automation or our human testers. These videos make it easy to understand failures and recreate bugs because you can see exactly what actions the tester (or automation script) performed and how the application reacted.
When watching replays of automated tests, for example, the test script will appear to the left of the video with the failed test step highlighted in red.
Clicking the “Investigate Action” button for any failed test step allows you to see the original element the test was searching for, the closest match the test found, what percent of a match it was, and more.
With manual tests, you’ll be given results for each browser or platform the test was run on. Then, for each browser or platform, you can click on individual test steps and view the recording of each tester.
All test results also include HTTP logs and browser logs. Further, manual test results include mouse activity, time spent on each step, and more.
Finally, Rainforest QA integrates with tools like Slack and Microsoft Teams, so you can get instant notifications for any test failure. If you integrate with Jira, every time you need a bug to be fixed, you can automatically create a ticket for the development team that includes a screenshot from the failed step and a link back to the failed test results in Rainforest.
This dramatically reduces the amount of time it takes to triage bugs—both for automated tests and manual tests.
Talk to us about the fastest, on-demand manual testing
8 Other Applause Alternatives & Competitors
Some teams turn to Applause to supplement their manual testing and some teams also take advantage of Applause for writing automated test scripts. Some of the top alternatives for Applause only offer manual testing, others only offer automated testing, and a few offer both.
This list includes a mix of options to help you make an informed choice. However, if you want to dig deeper, you can check out our in-depth comparison of manual testing options and our detailed guide to automated testing tools for web applications.
Testlio is one of Applause’s top competitors. They offer the option for a managed or co-managed manual software testing service for web and mobile applications. If you choose the co-managed service, you’ll be given the option to create and run some of your own tests. Either way, a few people from Testlio sit in on your team meetings until they understand your software product and development workflow well enough to create and run tests for you.
Once onboarding is complete and you’re ready for testing, an average testing session for Testlio takes 1 to 4 hours. They also advertise ‘overnight’ functional testing. Additionally, user reviews suggest that you may need to submit your testing suite one or two days in advance if you want to run a full suite of tests (such as your regression suite).
Like most of the manual testing companies in this list, Testlio offers in-the-wild testing, and has no pricing information on their website.
Test IO is a manual testing service that advertises their ability to make use of non-business hours by offering overnight and weekend testing. They start with a less-than-24-hour onboarding process to help you learn the platform and design your test cases, but the Test IO staff quickly move to a hands-off approach. For an extra fee, they will help you with long term test project management.
If you opt-in for the Test IO Elite package, they will return results within two hours—which is their fastest option for testing.
With any package, their goal is to minimize the number of bugs your team rejects over time. If you choose to add on the managed service, the Team Lead assigned to you will preview all reported bugs and reject any they think are irrelevant on your behalf.
Mindful QA boasts of being 100% located in the United States. They build partnerships that promote sustainability and eco-friendly practices.
Mindful QA strives for full test coverage of your web application with the idea that it will result in a bug-free environment—and a better digital experience for your users. They typically get started on testing within two days of beginning the onboarding process and keep you updated on when to expect results.
While you do have to sign a contract to use their service, you won’t be committed to using a specific amount of testing hours.
TestingXperts will create and execute an entire testing plan for you. They keep you apprised of their progress with daily, weekly, and monthly reports. The reports include details about which tests were executed and what bugs were found to help you monitor your product’s digital quality. All reports can be found on a customized dashboard.
Additionally, each test they create can be traced back to specific business requirements. This is how they keep the test plan accountable to your testing needs. Each test is run by testing experts from the United States, Europe, and India.
Finally, TestingXperts offers many types of testing including A/B testing.
Perfecto, formerly TestCraft, is a codeless testing solution with a built-in testing grid for mobile and web testing that uses artificial intelligence and machine learning to automatically generate Selenium test scripts (or Cypress scripts via their Cypress integration). The AI also has the ability to fix some broken tests during a test run.
Perfecto also offers:
- A reporting and analytics page where you can view heatmaps, graphs, and more
- Cloud-based device testing
- Remote bug fixing capabilities
For the test scenarios where you need to insert additional code, TestComplete offers keyword testing that lets you search for and insert actions based on preset keywords.
TestComplete also offers an object recognition engine which allows you to test elements inside a PDF or chart.
A lot of testing teams use the SauceLabs testing grid to run multiple code-based tests at the same time, however, SauceLabs also provides a low-code way to generate Selenium, Appium, Espresso or Cypress test scripts. To use SauceLabs’ test automation solution, you upload spreadsheets of detailed actions and datasets written in plain text and the testing platform generates the test scripts for you.
SauceLabs also provides Android emulators, iOS simulators, and real devices for ‘in-the-wild’ testing. These devices can be accessed through a private cloud.
SauceLabs supports mobile and web app testing, API testing and testing IoT devices.
UserTesting is less about testing to make sure your software is functioning properly, and more about evaluating the experience of real users.
To use the service, you choose a vertical and the exact type of customer you’re targeting — or choose to use an existing customer from your CRM — and they find matching users for you. Then, the tester moves through your application as their screen and a video of themselves narrating their movements gets recorded.
This type of usability testing gives you insight into real users’ experiences but doesn’t really support scripted, functional testing.