“Rainforest gives us the ability to increase the number of tests that we’re running without impacting our deployment speed.”
- No In-House QA: mindbodygreen’s CTO, engineers and product team all pitch in to keep quality high
- 70% Crowdsourced Coverage: About 70% of mindbodygreen’s regression testing is executed via Rainforest
- 2x Faster Deployments: Time to deployment is nearly twice as fast with Rainforest
mindbodygreen is a New York-based lifestyle media brand dedicated to wellness. Their team, which includes 4 developers, 2 product managers and a few graphic designers, maintains both media and ecommerce components of their website. As a result, the team often had to pitch in with the QA process, which pulled them away from other projects. Everything from discovering and cataloging bugs, to knowing what to test during each project, was a time-consuming chore. “We were invariably spending either way too much time testing, or not enough time testing,” says Tim Glenister, CTO of mindbodygreen.
During planning for a major revision of their media site, mindbodygreen realized that they wouldn’t have enough time to write code and run through their QA process. mindbodygreen turned to Rainforest to help alleviate the burden of testing on their team.
Writing Tests Earlier for Faster Deployments
With Rainforest, mindbodygreen can begin functional testing as early as possible. Tim begins writing tests as soon as the UX wireframes and functional specs are done. As a result, Rainforest tests can be mapped to code as it is written, allowing mindbodygreen to run functional tests almost as quickly as the engineers write code: Tim told us about their testing process with Rainforest:
“We’re testing our UX and UI almost at the same time as we’re running unit tests because we’re able to turn pull requests immediately into QA pushes and into QA tests from the specs. This helped us to find bugs quickly, especially across devices and browsers, and iterate through the specs much faster than we could before.”
Rainforest has allowed mindbodygreen to significantly speed up their development cycles. Two years ago, a redesign of one portion of their site took six weeks of grinding to complete: “It was a lot of nights and weekends of not only our engineers’ time, but our creative team and our product team that had to lend a hand with our QA testing. We were scrambling to find native devices we could test on,” Tim told us. Now that the team is using Rainforest, Tim says that a similar redesign was accomplished this year in just three weeks: “We were able to cut our engineering and production effort by about half by leveraging Rainforest.”
A Shift in Development Mindset that Helps Improve Quality
Tim noted that implementing Rainforest has also had a positive impact on how his team thinks about quality. mindbodygreen recently made the shift to having all front-end engineers write their own tests in Rainforest. As a result, engineers are thinking about testing during the development process, and have adopted a greater sense of ownership over the quality of their code.
Because Rainforest test results provide fast, consistent feedback to the developers, mindbodygreen’s developers are able to stay focused on quality without losing time to manual testing. Tim says, “Rainforest has had an impact on our code quality because it provides a continuous feedback loop. If we’re consistently getting failure reports on an issue, that’s a mental driver to fix the issue. Our developers want to have their code “green” and have all their tests work every time we run our test suite.”
Tim also notes that Rainforest has helped bring the test-driven development principles to the backend to the front: “On the back end, we always thought about test-driven development, but our front end has been a little harder to test. Rainforest has allowed us to write our regression tests alongside unit tests. It really helped us to find bugs quickly and push the product out faster than we ever have.”
Increasing Test Coverage to Release Confidently
mindbodygreen’s goal is to minimize the amount of manual tests that their team must execute by leveraging a combination of Rainforest QA and automated Ghost Inspector tests. The team has been able to increase coverage while reducing the time spent by the team on testing. “We can have hundreds of tests run in very little time,” says Tim. “We keep broadening our smoke tests every time we do a build. Rainforest gives us the ability to increase the number of tests that we’re running without impacting our deployment speed.” Tim told us that approximately 70% of the mindbodygreen’s regression tests are now executed via Rainforest.
Another benefit of implementing Rainforest has been a boost in confidence during releases. Tim’s team no longer has to worry about whether they’ve tested the right user flows or across the right browsers. Tim says: “Rainforest has increased our developers’ ability to get code out quickly because they’re more confident that it’s right. It has helped us catch less obvious issues that we didn’t catch in the development process.” As a result of their increased confidence in releases, mindbodygreen is able to move faster and keep their team focused on development.