Doing QA well is tough, even for the most innovative teams. Effectively scaling good QA is even tougher. The trickiest piece of this puzzle is designing a QA strategy that meets the needs of a speedy development team –while simultaneously adapting QA to support ever-changing business goals.
However, it’s not impossible to ship high-quality code quickly and tie your QA strategy to business goals at the same time. Our CIO, Derek Choy, recently sat down for a webinar to discuss how the key to doing QA well and scaling it along with your organization’s growth, lies in knowing how to scale QA processes without scaling your QA team.
3 Things to Consider Before Scaling Your QA Team
While there are several ways to approach scaling QA, growing a team of people is very common. Keep in mind that growing your team may not be the best option for your unique organization at this time.
Derek emphasized considering these three downsides that come along with growing a QA team when you’re strategizing scaling software quality:
- Bigger teams are harder to manage: People management resources are necessary to make a QA team successful, and the more people there are on the team the harder it is to manage. Having a big team means that more of your time is spent on making each team member successful with execution, which means less time is available for you to focus on strategic initiatives.
- Inflexibility for scheduling and capacity: People are inflexible. And for good reason! People have lives outside of work. Scaling your QA team can make managing QA on the weekends and at night a bottleneck to continuous delivery. While offshore teams are an option, it takes a long time to find people and get them up-to-speed on execution. Additionally, offshore teams often come with management difficulties.
- Growing a QA team can be expensive: Recruiting and on-boarding new talent cost a pretty penny –as does attrition. Headcount also does not come cheap. Scaling your team can be one of the most expensive ways to approach scaling QA.
Don’t jump the gun on growing your QA team when scaling your QA strategy. Instead, be strategic with hiring and consider other options to QA strategy scaling success.
Seed Stage: Stay Lean & Leverage Customer Feedback
Focus On: Functional Testing, Dogfooding, Headline Case Testing
In the consumer app market, the users of a seed-stage company’s product are there for the thrill of getting their hands on an early release. They expect the core functionality to work, but aren’t looking for all the bells and whistles to be there. That means you don’t need to worry about any of the edge cases, or even the kinds of issues exploratory and informal testing might surface. Even integration testing has limited value at this point; your product is going to go through so many iterations and change so significantly that these tests will mostly be wasted effort.
Focus on internal quality metrics to ensure that building more functionality into later releases won’t mean rewriting everything you’ve done so far. Test-driven development, unit tests and code reviews are key. Use your own product team to the greatest extent possible to verify functionality (dogfooding your own product is key to understanding headline cases at this stage. It’s also beneficial to realize that the early adopters who use your product now will understand it’s barely beyond beta, and will effectively do functional testing for you. Keep communication channels open with your early users and solicit feedback often.
If the product’s functionality is brand new, even a minimum working release will leave users hungry for more. But if you’re entering a crowded market, your product needs to have something that makes it stand out—in a good way. Because these features are so new, and so new to your product, they’re likely to contain serious headline cases that will impact your product’s functionality. More importantly, they will impact the perception of your product in a crowded market. Make sure you probe for any headline cases in your most novel features; doing so may mean the difference between major positive press for the unique value your product brings to the consumer, and major negative publicity that will stop your momentum before you’ve made it out of the gate.
Series A/B/C: Focus on UX for a Broader Audience
Focus On: User Acceptance Testing, Regression Testing, Integration Testing
At this stage, companies have released successful early versions of their product and found a ready customer base. The next goal is to build on that with a solid product that will start getting wider acceptance in the marketplace. Now’s the time to start including regression tests to make sure functionality doesn’t get broken from one release to the next. It’s also time to add integration tests.
Since you’re starting to have mass-market usage, your product should be pretty stable in its structure. Making sure system components hook together properly will ensure basic issues are found and eliminated before the app gets to user acceptance testing. You also need to start user acceptance testing at this stage. You can do exploratory and informal testing if you want, but to end up with a product that’s ready to be used by millions of consumers, you’ll want a more structured UAT process that covers the major usage scenarios.
Implementing a scalable QA process is important at this stage of growth. Make sure you have a plan for tracking and addressing the bugs that are identified so your next releases can draw in even more customers.
IPO: Prepare for the Masses
Focus On: Load Testing, Penetration Testing, Edge Cases
Once you’re ready to go public (or have already gone), you’re playing with the big boys. That means matching the quality levels of a major corporation. You don’t necessarily need to be Six Sigma, but if you want to be part of consumers’ daily lives like Facebook or Instagram, you need to be up and working when the customer is.
That means adding load testing to make sure you can handle the additional capacity demands. Protecting customer’s personally identifying information is also important, as your larger customer base will likely make you a more tempting target for hackers. Depending on the nature of your business and the kind of information you collect and store, penetration testing might be valuable to you. Even without penetration testing, testing application security features is important now.
Now that you have a reputation to protect, check potential edge cases in addition to the headline cases. Some of them may be important enough to matter at this point (you’ll need to decide based on your understanding of your business and your customers). Besides the formal acceptance test, bring back exploratory and informal testing. You can afford to spend time on that now, and the “just poking around” method of testing can surface issues that “comprehensive” test plans miss.
Leverage Tools to Automate the QA Process to Scale Your QA Strategy
Leverage as many useful tools as you can to improve the execution of QA as well as the QA process. – Derek Choy
Implementing tools to automate your QA process is an effective way to scale QA without scaling your QA team. A QA solution built for a fast-moving development process will be more affordable than growing headcount. While giving your team everything it needs to bring the QA process up to speed with your organization’s growing business goals.
QA strategy and execution solutions like the Rainforest 3.0 Platform empower teams to take a strategic approach to software quality and automate the QA process. By filling the QA coverage gap, the Rainforest 3.0 platform increases product quality, customer satisfaction and revenue goals.
Learn the Role of Test Automation in Successful QA Scaling Strategies
I think test automation should be a part of every QA strategy, but not all of it. – Derek Choy
Automating QA testing can speed up quality feedback and accelerate time to software improvements, but it’s not without limitations. Test automation contains a range of hidden costs, from hiring engineering talent to script and maintain tests, to maintaining the environments automated tests run in.
The key to successful QA test automation lies in being strategic with which tests you decide to automate. Remember that every organization is unique, so there is no one-size-fits-all answer to how much to automate.
Don’t Expect QA at 100 People to be the Same at 10
If you’re one of the rare organizations that nailed QA in the early days, it’s tempting to stick with it as the company grows. But staying the same while teams and products are evolving and changing is a sure-fire way to collapse your QA success. A QA process — even a good one — should adapt with your team as it scales.
