When your organization has over 2 billion lines of code and 25,000 engineers, how do you keep quality high?
As Google has grown, they’ve had to create a QA strategy that keeps up with the demands of an organization with a huge number of moving pieces to keep track of — from collaborative teams across the world to a shifting landscape of new features and products — while enabling their team to continue innovating across the board. They’ve managed to do so with a relatively small number of dedicated testers for an org of their size.
Want to learn some lessons from Google’s QA at scale? We dove into the archives of the Google Testing Blog to discover how other growing teams can learn from the way Google approaches software testing.
Think Small and Fast
Just because Google is big organization doesn’t mean that everything they do is big — when it comes to testing, Google favors small, frequent releases and small, concise tests. This incremental approach allows them to better control changes to the user experience over time and pinpoint issues efficiently. As a result, they invest heavily in unit and integration testing and use end-to-end testing more sparingly. In general, Googlers are encouraged to look for ways to speed up testing cycles and minimize test failures to ensure that the team’s QA process is as efficient as possible.
Software Testing is a Focus, Not a Team
As Google’s team grew, software engineers building and testing their own didn’t scale. Instead of rolling out a more traditional testing team, Google started creating specialized engineering roles to help boost the organization’s QA bandwidth holistically.
Now there are several engineering roles within the “testing” focus to solve different quality problems: Software Engineers, who write test code; Software Engineers in Test, who focus on testability and refactor code to make it more testable; and Test Engineers, who organize overall test strategies and write automation scripts.
But despite these specialized roles, there’s no “over the wall” QA process. “At Google it’s the product teams that own quality, not testers. Every developer is expected to do their own testing. The job of the tester is to make sure they have the automation infrastructure and enabling processes that support this self reliance. Testers enable developers to test,” wrote James Whitaker in his series on How Google Tests Software
Communicate Effectively
Part of making this developer-owned testing approach work at scale is to prioritize communication channels. Engineers who focus on testing aren’t siloed in their own areas of Google’s offices, but are integrated with the rest of the engineering team.
Google also puts an emphasis on communicating product quality internally to increase ownership and spur the team to focus on quality every day. For example, Google has internally published data about code coverage helped increase adoption of best practices.
“The feedback from our fellow engineers was overwhelmingly positive. The most loved feature was surfacing the coverage information during code review time. This early surfacing of coverage had a statistically significant impact: our initial analysis suggests that it increased coverage by 10% (averaged across all commits),” wrote Marko Ivanković in a post on measuring coverage at Google.
“Testing on the Toilet”: Foster a Culture of Quality Ownership
Perhaps the most eccentric QA practice at Google is the long-running tradition of posting fliers — featuring a “quiz that changes every few weeks and asks technical questions about testing programming code for bugs” — on the walls of bathroom stalls at HQ. This practice, dubbed “Testing on the Toilet”, helps keep software testing best practices at top-of-mind for the team as they go about their day. While this quirk isn’t quite a pillar of Google’s QA strategy, it does signal something important about the way the org thinks about quality ownership and best practices.
Finding the Software Testing Strategy to Help You Scale
Whether they’re large enterprises or small startups, every organization must find an approach to quality that will fit their team’s values and workflow. Finding successful role models for everything from how your team communicates about quality to the tools and platforms they use to get the job done can help you rethink your approach to QA and start seeing results!
Want to read more about QA innovators? Learn how growing teams like SolarWinds are using Rainforest at scale.
