Test automation can be an effective piece of the QA strategy puzzle for any team looking to optimize their testing process. Done well, it can speed up software testing cycles, improve the efficiency of QA teams, and ultimately increase product quality. But contrary to popular belief, test automation is not a magical solution to achieving cheap, lightning-fast, effortless QA results.
In a poll survey conducted during our webinar on the cost of test automation, 58% of quality owners reported the cost of test automation to be significantly more expensive than initially forecasted. Rainforest CIO Derek Choy explained two major reasons that a team’s test automation initiative can be pricier than they expected.
1. Test Automation Requires Ongoing Engineering Resources For Maintenance
Many QA teams have the impression that automation is relatively low cost and efficient. But in practice, test automation can actually be quite expensive. Derek stressed that in order to forecast and evaluate the true cost of test automation, teams must take three cost areas into consideration: test writing costs, environment maintenance costs, and test maintenance costs. This final area — automated test maintenance — is often overlooked, but in the long run represents a large amount of the resources needed for test automation to stay effective.
“One thing people should not take lightly and forget, is the cost of maintaining automated tests.”
Automated tests require technical expertise to continually manage tests and the environments in which they run. Test automation experts are highly sought-after, and often demand competitive salaries. As a result, hiring out a team to continuously keep automated tests aligned with the code can be a significant expense.
Application feature changes are made sprint-over-sprint. When these changes are directly connected to automated tests, the tests can break. Additionally, when application changes are made to features that indirectly connect with automated tests, the tests can break. When evaluating the cost of incorporating automation into your QA strategy, it’s extremely important to consider how frequently automated tests will need updating to stay useful.
2. A Hidden Cost of Automation is Delayed Release Timelines
Along with the cost of technical talent to write and maintain automated tests and their environments, there’s also the cost of delaying new software releases. A robust automation strategy essentially creates an additional codebase that your team must maintain. This is especially problematic if you have developers maintaining your automation test suite, as the time they use to refactor broken automated tests is time they can’t spend creating new features.
Triaging failed tests can also create delays. Automated tests can fail because there is a bug, but they can also fail because the test itself is broken. If your team is in the middle of launching a release and your automated tests fail, you’re not likely to continue with the release until the source of the failure is identified, because there might be problems with software quality.
Crack The Automation Code By Learning How to Create a Balanced Test Automation Strategy
While test automation can be an excellent tool for QA teams to streamline testing, the cost of maintaining automated tests is not trivial. The key to realizing a high return on your investment in test automation is to determine which tests cases are a great fit for automation, and which are best suited for manual testing.
