Developing a QA strategy that works well is challenging. Developing a QA strategy that continues to work well as your organization scales is even more difficult. In our new guide, Agile QA at Scale: Software Testing Strategies from Enterprise Teams, we dig into lessons from successful QA teams at enterprise organizations, including Google and Facebook, to learn how they’ve designed QA strategies and processes for quality at scale. There’s no one-size-fits-all QA strategy that will work for every team. But there are a few key traits that successful QA teams have in common.
In order to scale QA without scaling a huge QA team, automation is key. Enterprise products require a large amount of testing that often isn’t feasible without some type of automation. But test automation can easily become a black hole of development resources and budget. Relying too heavily on automation can yield poor returns. The most successful QA teams use automation strategically to ensure that they get all the benefits of automation’s speed, while minimizing test maintenance and QA debt.
“At Spotify, we want the manual testing effort to be focused as much as possible at feature testing, less on regression tests. But we still have to do regression. So, we tried to automate a big chunk of that. Regression tests are run on our Desktop, Android, iOS and WebPlayer clients, and also some backend services.”
- Kristian Karl, Test Manager for Spotify
Quality doesn’t happen organically. Some organizations integrate QA functions into other teams -- such as product or engineering -- to keep the team lean. But this “no-QA” approach to quality may let issues slip through the cracks. Even for teams with an in-house QA team, clear quality ownership throughout the application development process can be unclear.
Highly-efficient teams ensure that quality ownership is always clear, and testing roles are always well defined. For instance, Atlassian expects developers to own the quality of any code they create, but the QA team is always available to help devs stick to best practices. This practice empowers developers to write better quality code and helps resolve issues earlier.
Many parts of the organization influence quality, from the designers and PMs who create product specs, to the developers to implement them, to the customer success team that gathers feedback from users. High-performing organizations make quality a focal point for every team and individual, no matter what their role. Google takes their commitment to quality to the next level, by making sure that on-campus bathrooms have a supply of QA-related reading material on hand.
Measuring quality can be tricky, and there are plenty of red-herring metrics to lead you astray. Teams often begin by measuring metrics that focus on input, such as test cases written. By contrast, teams that have found a balance between scaling their QA results and keeping their team and processes lean keep their focus on measuring overall quality.
Learning from the successes (and failures) of enterprise organizations can help growing teams understand what they’ll need to scale effectively. Download the guide now to find out what your team can take away from how Facebook, Google, and other teams approach QA.
A comprehensive overview of DOM-based automation's limitations, and why UI testing may be superior for most testing use cases.
The landscape of software testing is changing. Speed and quality are no longer seen as opposing forces.
Software development and QA practices are at the heart of the show Westworld. Check out the five lessons real QA teams can learn from the programming team at Delos, Inc.
Rainforest CTO, Russ Smith, discussed testing and troubleshooting mobile application with AWS Device Farm at AWS re:Invent this year.. Watch the full video here.