crowdsourced testing

QA as Software, Not Service: How Rainforest is Different from Other Crowdsourced Testing Solutions

Picture of Ashley Dotterweich
Ashley Dotterweich, Wednesday September 6, 2017

In a recent post, Rainforest QA co-founder and CEO Fred discussed how the acquisition of Applause can negatively impact their customers. Crowdsourced testing is a new and innovative approach to QA, and for many development teams, adopting a crowdtesting solution represents a significant shift in the way they think about their QA strategy.

Many people ask us how Rainforest QA is different from the other options for crowdsourced testing. See why Rainforest ranks as one of the most innovative software testing platforms on the market at G2Crowd. Today, let’s dive into how Rainforest compares and why that difference matters.

QA as a Software vs. QA Services

“Other crowdtesting solutions gave us access to a lot of testers, but not in a smart way. They needed a lot of support on the project, without us gaining any more efficiency over time. By using Rainforest, we don’t have to ramp up our QA resources continuously. That saves us a ton of money.” - Dominic Esposito, Jitjatjo Head of Product

The model for most crowdsourced testing solutions is, more or less, a distributed outsourced model. For companies like Applause, tests are managed by a dedicated account manager who communicates with, vets and manages the workflow of the testers who are executing tests. This approach effectively outsources management of the tester network as well as execution. While this model of crowdsourcing work can be useful for teams with limited resources, it has some drawbacks. Communication breakdowns are still common, and time-to-execute can still drag down feedback time. Because all test results must be funnelled through a test manager, this outsourcing to the crowd doesn’t scale up very efficiently without dedicating additional resources to testing.

More than Crowdsourcing: Machine Learning for QA

Unlike other crowdsourced testing solutions, the work done by Rainforest’s crowd of testers is managed using machine learning quality checks. We don’t have test managers because we use machine training data with over 78 million data points to manage test execution, giving users weighted results they can trust in 30 minutes. trust at a speed that lets them stay competitive.

Rainforest QA vs Crowdsourced Testing Services

Our model of machine learning-powered crowdsourced testing makes a big difference in the results that Rainforest customers see.

Faster Results

“We can have hundreds of tests run in very little time...Rainforest gives us the ability to increase the number of tests that we’re running without impacting our deployment speed.” - Tim Glenister, mindbodygreen CTO

Because execution isn’t dependent on a specific testers or account managers, Rainforest users can kick off tests at any time and start running tests without any delay. We focus on optimizing user test suites to be executed quickly and efficiently, and execute test runs in parallel to ensure that our users get test results in 30 minutes.

Consistent Execution at Scale

“Rainforest took the crowd and turned them into a reliably executing test engine. It gives me the assurance that the tests are executed exactly the way we meant for the them to be executed. It’s a highly reliable process.” - Daria Mehra, Quid Director of QA

In addition to being able to initiate tests at a moment’s notice, Rainforest provides consistent testing environments across every single test run. Many other crowdtesting solutions focus on “in-the-wild” testing, which leverages real user devices to execute tests. But while getting feedback from real-world environments is valuable, it doesn’t provide the consistent backdrop for functional and regression testing that teams ultimately lead.

At Rainforest, our combination of virtual machines and device labs provide a clean slate for every single test. Since these VMs are spun up on demand, availability of testing environments is never an issue. Rainforest offers guaranteed access to every browser in our offering, at any scale. As a result, pinpointing issues and resolving bugs can be achieved more efficiently.

Workflow-Native QA Testing for Developers

“Our devs realized that they wouldn’t have to leave their code editor to write a test for something they had just developed. It was a lightbulb in all our heads that allowed us to shorten the cycle between writing code and writing the tests that go along with that code.” - Mitch Stewart, Guru CTO

QA processes often sit on top of, adjacent to, or “over the wall” from development. At Rainforest, we believe that continuous testing is the future of product quality. As such, we’re focused on integrating QA testing into the development workflow.

Rainforest is designed for developers to use. For teams with developer-owned quality processes, Rainforest provides a testing solution that integrates well with engineering workflows. Rainforest not only integrates with your dev team’s favorite tools — including CI servers and bug trackers — but also allows developers to write and execute functional tests from the command line. No other solution offers a QA execution process that functions like software and fits seamlessly into the development process.


Rainforest is the fastest crowdtesting solution on the market, returning complete, reliable results in about 30 minutes. For teams that need to develop faster and stay competitive in a crowded market. there is no comparable alternative. Want to see the difference for yourself? Learn more about the results that Rainforest delivers to our customers.

Filed under: