Do your QA goals include automating as much as possible, as soon as possible? Does that goal seem like it’s always just beyond reach? If so, you’re not alone. According to the World Quality Report for 2016-2017, 41% of agile teams still struggle to apply automation at the levels that they would like to.
Teams that aim for 100% automation often quickly discover that managing a robust suite of automated tests can become a major resource and time sink. While testing automation seems like it should be able to keep pace with fast-moving development cycles, testing automation strategies often fall short of the mark, relying on testing practices that bottleneck deployment just as much as manual testing.
In this post, we'll explore how using Rainforest to crowdsource QA testing provides an alternative to testing automation that empowers teams to test faster and get better results.
For many teams, the gap between tests that must be executed manually and tests that should be automated is wide. A test may need to be executed over and over again for functional verification long before the feature is solid enough to merit an automation script. Crowdsourcing these repetitive but necessary tests can help teams bridge this gap at scale.
The development team for marketing automation software Bluecore has a goal of having 80% of test cases automated before a feature reaches general availability. However, frequent changes and updates to their UI made maintaining this level of stable automation tests challenging.
Co-founder and CTO Mahmoud Arram shares that by leveraging Rainforest, his developers are able to write Rainforest tests for the UI that are flexible enough to be executed and updated rapidly. “On the server side, they’re writing automated [unit] tests. On the UI side, they’re building a simple Rainforest test that exercises the happy path of the feature. This way changing the code and then changing the test isn’t so arduous,” says Mahmoud. Testing automation is still the ultimate goal for Bluecore’s testing strategy, but Rainforest allows the team to develop and test faster before test cases are ready for automation. “Our engineers can work on developing new features and not worry about regressions on the old ones, because our Rainforest tests will catch them,” he adds.
Since automated tests are prone to breakage, QA and development teams must have a strategy for addressing these issues as they come up. Crowdsourced testing allows teams to “roll back” broken tests to a more manual execution method without losing some of the benefits of automation -- specifically, fast, hands-off test execution. Because the test continues to execute even if the automation script is broken, dev teams don’t lose coverage because of flaky or brittle tests.
Teams that leverage crowdtesting to speed up their dev cycles also free up resources for high-value activities -- whether that’s UX testing or development new features. In a webinar on legacy application testing, NPR’s Paul Miles commented that testing automation often incurs a number of hidden costs: _“Where a lot of people go wrong who are focusing on these UI type tests is underestimating the cost of managing any kind of test automation framework. If you use a tool like Selenium you’ll find out quickly that you have to hire a test automation engineer or take someone within your own development department and kind of task them with developing and maintaining this these this test automation framework.” _
The World Quality Report also noted that functional testing automation expertise is one of the most challenging skills to hire for this year. By using automation more strategically, teams can ensure that their time is spent on automating stable test scripts and other strategic QA activities, rather than scrambling to fix broken tests and executing manual tests.
Want to learn how some of the best testing automation experts in tech are thinking about automation strategies right now? Download the DZone Guide to Automated Testing: Improving Application Speed and Quality for more on how to get more out of your automated QA strategy and get beyond the basics of Selenium usage.