QA can be a challenging aspect of development to manage because of its variability. Uncertainty about how much testing to do, time-consuming test execution and the drawn out cycle of finding, fixing and retesting for bugs all contribute to a sometimes lengthy process.
One of the most effective ways for teams to bring their QA processes under control is to "shift left" and start testing earlier. From thinking about UX testing from day one to implementing processes to speed up the feedback loop on code quality, development teams are finding new ways to leverage testing earlier in their development process. Here are three ways that three different companies have "shifted left" to help speed up development and deploy more confidently.
The practice of test-driven development (TDD) -- writing tests first, then writing to code to fulfill the test -- helps many teams create quality-focused applications and minimize bugs. But it’s a process designed for unit tests, rather than feature-level testing. For most companies, UX testing doesn't happen until late in the dev cycle, which can create slow down the overall development process. In order to create a more focused UX testing strategy, lifestyle media brand mindbodygreen has applied TDD practices for their front end team by using Rainforest. CTO Tim Glenister told us:
“On the back end, we always thought about test-driven development, but our front end has been a little harder to test. Rainforest has allowed us to write our regression tests alongside unit tests. It really helped us to find bugs quickly and push the product out faster than we ever have.”
Mindbodygreen writes their UX tests while their developers are working on the code. As a result, they can run these tests as soon as the code is ready. “We’re testing our UX and UI almost at the same time as we’re running unit tests because we’re able to turn pull requests immediately into QA pushes and into QA tests from the specs,” Tim says.
Another way of integrating testing into the development cycle is to run a suite of “health check” tests on a regular basis. Teams deploying code very frequently have an increased risk of introducing new regressions into their code base. Running a selection of critical tests throughout the development cycle can help mitigate this risk and spot bugs more quickly.
The team at solar energy management platform Sighten develops code in two-week sprints. But the process of discovering, fixing and retesting for bugs often became a bottleneck to meeting their biweekly deployment deadlines. To catch bugs quickly, Sighten began running a small suite of key functional tests every day. “We automatically run our Rainforest test suite every morning on our development servers and use that as a health check of how development is going for the sprint,” says Mariya Nomanbhoy, VP of Product. “It really gives us an idea of what could be going wrong and whether any new regression issues have been introduced since our last check.”
Adopting this practice allows Sighten to catch bugs without creating additional work for the team. Because they find issues more quickly, Sighten’s team has been able to reduce the amount of time they spend dealing with bugs overall.“Before Rainforest, we had been spending 50% of our time [during each sprint] on bug fixing,” says Mariya. “Now we only spend around 15% of our time fixing bugs.”
Earlier testing can also mean exposing new features to your team earlier. In addition to running Rainforest regression tests, online art marketplace Twyla has empowered their entire team to become quality owners. Twyla holds weekly “Test Fests,” where everyone from designers to sales team members are invited to come learn about new features and participate in manual testing. This allows the product team to socialize new features with the team. “In a way it’s our version of an internal demo day. Everyone can come and see new progress from the development team, ask questions, and get up to speed on the product,” says John Fitch, product manager at Twyla.
Involving the team in testing while features are still in production helps give everyone a stronger sense of ownership over Twyla’s product. Quality becomes a company-wide concern, rather than a specific job function. The process also gives the team a more efficient way to communicate any issues they encounter to the development team.
For each of these three testing use cases, testing earlier gives the development team more confidence in the quality of their code. As a result, these teams are able to ship code faster with less risk of breaking something. For some, like Sighten, that means hitting deadlines more easily. For mindbodygreen, tester earlier means reducing their time to deployment by up to half.
Want more on the strategies that Sighten, mindbodygreen and Twyla are using to speed up development and ship higher quality code? Learn more on the Rainforest customers here.
You can only move as fast as your QA process allows. That’s why in order to do continuous delivery, you need to adopt an equally continuous QA process.
How the Rainforest team ships code, and what we're doing to make our deployment strategy better, faster and stronger with continuous delivery.
You probably heard the news, CI is cool. In this post I'm going to walk you through the basics of what CI is, and why you need to use it, like now.
A less obvious cost is your CD development is effort required to keep your application CD-friendly. This post covers one of the thorniest problems of the continuous delivery world: deployment race conditions, and two alternatives for solving this issue.