How Do I Design a Customized QA Strategy?
There’s no true one-size-fits-all QA strategy. Teams that customize their QA strategy to map to their business goals and team’s needs are able to use testing resources as efficiently as possible.
Why You Need a Strategic Approach to Testing
Many teams take an ad hoc, reactive approach to quality. They test what they think to test, usually only when there is an immediate need to test. But this approach leaves your team constantly putting out fires, rather than getting ahead of them. Without a strategy to frame QA initiatives and drive quality forward, testing becomes a time-consuming, resource-hogging activity without increasing ROI over time.
A good QA strategy allows QA to become more proactive and more efficient over time. It also provides a focused, repeatable process that ensures that the team is confident in code quality for every release.
The Strategic QA Checklist
- Start by designating a QA Owner.
- Get buy-in from the team, then communicate about the process and goals.
- Start small, but set up for scaling.
- Measure progress and set a cadence for reporting to stakeholders.
- Revisit every 6-12 months.
- Gather input from peers and experts.
5 Steps to Implementing a QA Strategy
1. Evaluate Your Process
Audit your approach to QA, from high-level goals (if you have them for your day-to-day process). The key here is to look for gaps and inefficiencies.
Discovering Your Own Quality Goals
A QA strategy needs goals, and those goals should be aligned with your organization’s needs and goals. If you don’t already have quality goals in mind (or even if you do!), a good starting place for more strategic QA is getting a bigger picture of what matters to your business, other teams within the organization, and your customers.
Find key stakeholders from across the organization (as well as customer advocates on teams such as Product or Customer Success) to better understand their goals, and where they feel the biggest quality pain points. Dig into data sources for product usage and customer behaviors to help support these conversations.
5 Essential Questions to Answer
- Who owns quality within your organization?
- Are you currently measuring success with quality? (If yes, how?)
- What is your approach to test case management?
- How do you prioritize testing and coverage on your product?
- What challenges are you currently experiencing in product quality?
Now it’s time to identify your testing philosophy and set a game plan for implementing your new strategy. Your strategy should include:
- How QA activities align with business goals
- How your team will balance short-term wins, long-term goals, and laying the groundwork for scalability
- What you’ll focus on testing, and how you’ll execute those plans
- Where QA lives throughout the SLDC, including when it begins for a given feature
Keep the Initial Team Small, but Include Key Players
Your team is hopefully excited about a new QA strategy, and there’s likely a long list of people who want to be involved. Having too many opinions feeding into the process early on tends to create confusion and can slow down the overall timeline for implementing the new strategy. Keep the “Strategy Rollout” team small, but be deliberate about who you include. These key stakeholders are the ones that will ensure that when it’s time to introduce your plan to the rest of the team, the transition goes smoothly.
Determining your strategy and all the moving pieces that go into it usually takes some time. You’ll likely have to go through a few iterations until you find the right approach. As a result, it’s most effective to figure out your workflow before you roll it out as a complete process for the entire team.
Focus on testing smarter, not harder. A strategic QA process should be designed around building focused test coverage to maximize impact, rather than a large number of test cases. Prioritize testing activities that will provide the most valuable insights to the areas of your product that matter most, rather than getting the most insights overall.
Your testing process should be designed with your team makeup in mind. What does quality look like during development? Is it a consideration during product design? Do you have minimal automation resources, but many hands on deck for manual testing? Be mindful of how QA will be implemented not just before a release, but throughout the entire SDLC.
The execution of your QA strategy should take into account a number of goals. Finding a balance between your short-term goals (e.g. executing your smoke suite as quickly as possible) and long-term goals (such as reducing the average number of bugs in production for every release over time) should dictate how you execute your strategy.
No matter what your QA strategy involves, it’s critical to have test environments that mirror production as closely as possible. While running smoke tests against your production environment is important for features in GA, waiting to start testing new features until after they are released increases the cost and complexity of resolving any issues that may emerge.
Execution Tools and Platforms
You’ll need to determine what execution methodologies will work best for your team and your quality goals. Many teams focus heavily on test automation tools such as Selenium. However, aligning the testing methods you use with your larger goals is essential. A team that is adding new functionality to a larger, more stable codebase may want to employ automated tests for their stable features, but execute tests for newer, more dynamic features manually while they are still changing frequently.
Learn more about choosing your testing methodologies in our webinar on Cracking the Automation Code.
Determine what role you want any in-house QA testers or engineers to play. Aiming for as much hands-off test execution as possible when it comes to regression and smoke tests frees up your QA team — whether that consists of a single automation engineer or an army of manual testers — to focus on more complex issues.
5. Measurement & Reporting
Before you really get going, you need to identify the metrics that will help you stay on top of your goals. Make sure your QA system is set up to help you measure these easily. Metrics that are challenging to measure won’t get tracked. If you can automate measurement, all the better!
Just as important as measuring the right metrics is communicating around them. Schedule check-ins with key stakeholders to ensure that they are informed of your process and how it’s going. Don’t forget to include the larger team in your communication plan. If they’re working hard to implement a QA strategy, they’ll benefit from knowing what’s moving the needle and what’s not.
Learn More About Measuring Your QA
Not sure what to measure? Jump to our chapter on Defining Quality Metrics to learn what to measure and how to communicate your QA successes.
How Much Do I Need to Test? Creating a Custom QA Strategy
In this session, CTO Russ Smith demonstrates how teams can create a “rightsized” QA strategy that balances quality insights with fast turnaround.
Developing with Quality in Mind: A Blue Print to Power Up Your QA Strategy
In this session, veteran QA experts from the Rainforest team share their experience and advice on shifting your approach to QA to be more strategic and more impactful.
What is the Right Amount of Coverage?
Most teams struggle with finding the right balance of test coverage that provides confidence and quality insight. Too little testing can leave critical bugs undetected before launch, but too much testing can slow down your deployment process and create shipping delays. So how can you find the right amount of testing for your product?
Start by identifying the common user flows in your application. Most of the time, the common parts will be built upon to allow you to get into the right state for your application. This applies for both automated and manual test suites. If using automation, make sure to have a common library of states you can re-use to get into the correct state. If using manual testing, you will need to find a test case management system that supports composable tests.
Using Data to Define Your QA Strategy
To determine where to focus your test coverage, start by digging into your user data. Understanding where your users are spending the most time within your product will allow you to map your test coverage to features and product areas that have the highest traffic or provides the most value to your users.
Look not only for what features and product areas get the most traffic but how users are navigating it as well. Are they primarily focused on a specific device or browser? Weight your testing to follow the behavioral patterns of your users. This will help you get as much of your testing activity aligned with your users’ needs as possible, rather than “wasting” tests on edge case that impact a smaller percentage of users.
Start looking for this data in behavior tracking tools, such as Google Analytics, Heap and MixPanel.
To Build Your Best Test Coverage:
- Use product data to identify important areas
- Create test cases around features and key user flows
- Focus on keeping your test suite tight and maximizing impact
A Feature-Driven Testing Approach
Now that you’ve identified the features that you want to focus on, you’ll need to build out the right amount of test cases for those features. By shifting testing focus from overall coverage to strategic feature coverage, teams can drive more effective, impactful QA processes.
To alleviate the confusion and frustration of writing the wrong tests, we recommend taking a feature-driven approach to testing. Feature-driven testing is a pragmatic, feature-based approach for front-end testing. It can also be applied to other types of testing, such as unit testing.
Every test suite should include five smoke tests that test the most-common user activities. These tests should be focused around core feature flows that are essential to your application staying up and running.
Happy Path Tests
Beyond the core feature flows covered by your smoke tests, you should also write ten tests that cover the most common user paths. Again, these are central to most users’ experience of your product.
Bugs happen, and your regression coverage should ensure that they don’t happen where they will matter the most. For regression coverage, select the fifteen tactical breaking points in the feature to make sure that they always work.
What About Edge Cases?
Edge cases should be a minor part of your testing puzzle. By nature, they aren’t likely to impact a large number of your users. There is also an almost infinite number of edge cases that can be identified for given product, making “full coverage” of edge cases an impossible goal. As your user base grows and becomes more varied, edge cases may be a more pressing concern. But don’t worry about them when you’re building out your core test coverage.
Test Coverage that Works Smarter, Not Harder
Once you’ve defined those test cases, you will have a strong foundation for QA that you can have confidence in. By building out coverage in sets of thirty strategic tests at a time, you’ll be able to ensure that test coverage stays focused and concise.
Level Up Your QA Process
Want to empower your team to deliver better quality results? Sign up for updates on exclusive QA University content. We’ll only email you when new content launches.