Anyone who has built out a suite of QA tests knows that your database of test cases can get bloated fast. In the 2017-18 World Quality Report, 55% of teams stated that they have challenges managing the size of their test data sets.
For teams trying to move fast and use their development resources efficiently, this challenge is a major cause for concern. The larger your test suite, the more maintenance it requires and the more time QA will take away from your team. What’s more, testing “all the things” often creates more noise in your quality metrics and can create confusion around whether or not a feature is ready to ship.
By shifting testing focus from total coverage to strategic feature coverage, teams can drive more effective, impactful QA processes.
To alleviate the confusion and frustration of writing the wrong tests, we recommend that our users take a feature-driven approach to testing. Feature-driven testing is a pragmatic, feature-based approach for front-end testing. It can also be applied to other types of testing, such as unit testing.
The goal of feature-driven testing is to create a structured test suite that focuses test coverage where it is most critical. To build these strategic test suites, we use the 5/10/15 Rule to write the first 30 tests for any given feature or application.
Every test suite should include five smoke tests that test the most-done user activities. These tests should be focused around core feature flows that are essential to your application staying up and running.
Beyond the core feature flows covered by your smoke tests, you should also write ten tests that cover the most common user paths. Again, these are central to most users’ experience of your product.
Bugs happen, and your regression coverage should ensure that they don’t happen where they will matter the most. For regression coverage, select the fifteen tactical breaking points in the feature to make sure that they always work.
Once you’ve defined those test cases, you will have a strong foundation for QA that you can have confidence in. By building out coverage in sets of thirty strategic tests at a time, you’ll be able to ensure that test coverage stays focused and concise.
Note that you should stay away from testing edge cases, as they tend to provide the least value for the effort they require.
Finding the right amount of coverage that gives you the confidence to ship without slowing your team down with excessive test execution and management. To learn more about the 5/10/15 Rule and how to create a customized testing strategy, check out our webinar on How Much Do I Need to Test? Creating a Custom QA Strategy and this blogpost on how we designed our own testing strategy, Optimizing Rainforest for Rainforest.
A comprehensive overview of DOM-based automation's limitations, and why UI testing may be superior for most testing use cases.
The landscape of software testing is changing. Speed and quality are no longer seen as opposing forces.
Software development and QA practices are at the heart of the show Westworld. Check out the five lessons real QA teams can learn from the programming team at Delos, Inc.
Rainforest CTO, Russ Smith, discussed testing and troubleshooting mobile application with AWS Device Farm at AWS re:Invent this year.. Watch the full video here.