Rainforest QA Customer Spotlight: Megan Bruce for Curalate
In the Rainforest Customer Spotlight series, we highlight the superstars of the Rainforest QA platform: the Rainforest project managers that write tests, manage databases and help build better quality processes for their organizations.
For this post, we interviewed Megan Bruce, a release manager-turned-product marketing manager at Curalate who has been using Rainforest QA since May 2016. Read on to learn how Megan measures the success of Rainforest at Curalate, and her suggestions for creating the most efficient test suite from day one.
- Profile: Support Engineer > Release Manager > Product Marketing Manager
- Time Allocated to Rainforest: Approximately one full month during the onboarding phase to get our initial regression test suite up and running; moving forward from there, about 10-20 hours/week to maintain tests, add new ones, and review test results.
- Secret Sauce: Proving ROI of Rainforest to the broader development team to get them onboard
- Advice: Focus on your critical path first and then move on to getting granular with regressions. You’ll find a lot of value from your critical path and then can make a more data-driven decision on regressions to focus on next.
What does Curalate do?
Curalate enables discovery-driven commerce at scale. Through our platform, brands can ingest media from multiple platforms and sources; tag products to that media; and distribute it to touchpoints like on- and off-site shoppable galleries, email, and blogs, helping consumers discover new products and driving traffic and revenue. It’s really exciting to be able to work with some of the world’s largest brands, like Urban Outfitters, Nordstrom, Crate & Barrel, and Sephora.
What’s your current role? What was your path to getting there?
I started at Curalate as a support engineer. Being in support, I saw a lot of issues coming in that could have been caught with more robust regression testing. We had found out about Rainforest, and it seemed like a great option to be more innovative about regression testing, rather than investing in a team of manual testers.. I then stepped into a release engineering role and began managing our deployments as well as our Rainforest implementation.
I’ve now moved onto the product marketing team, which felt like a natural progression since I was handling a lot of our release and deployment processes, including writing release notes and assisting with product launch plans.
What does your QA testing process look like?
We currently deploy to our web app twice a week, and we’re working on increasing the frequency of those deployments. Before each deployment, we run our Rainforest test suite and make sure there are no issues. We prioritize testing the critical paths. We know that our clients have incredibly busy days and need to move quickly in our platform, and so we don’t want any critical-path issues slowing them down or causing a frustrating experience.
What other apps, software, or tools are critical to your QA process?
Rainforest is our primary tool for doing regression testing for our web app. We love it. In addition to Rainforest, we rely on a combo of unit tests, integration tests, and Protractor for some basic automated UI tests.
What does your average week as a Rainforest PM look like?
As a release engineer, I helped managed the deployment process, including deploying to a staging environment for regression testing, and then deploying the QAed build to production. I was typically able to determine within an hour of running our Rainforest test suite whether or not all tests had passed. If there were failures, I’d work with the developers to triage the issues and determine whether or not we needed to block the deployment for a fix.
In a typical week, I’d also spend time maintaining current tests, like updating tests based on any recent code changes or product launches.
What impact has Rainforest QA had on your product quality?
I noticed a significant drop in the number of critical-path bugs reported in production - the kind of bugs that previously would have required a developer to drop everything and fix. It’s really exciting to review a test and find a failure, because it means that Rainforest caught something that would have otherwise gotten through to our clients. It’s really been awesome to use Rainforest to catch that stuff without having to use manual time for our staff to find those issues.
The tester video feature really helps a lot. It’s so helpful to be able to play back that video and see exactly what the tester was hitting. We take any issue that’s reported pretty seriously, and if there’s an edge case we want to catch it. If we see one failure and there’s some weird error that popped up, we want to dig into why that happened. Looking back at the video allows us to reproduce the issue easily when needed.
What was your secret sauce that helped you ensure the success of Rainforest at Curalate?
Demonstrating the cost-efficiency. The fact that I’m able to tell the team that we’ve been able to cut the regression testing time from 5-6 hours of manual testing to <1 hour with Rainforest helps demonstrate the massive time-savings of using Rainforest. Every time there is a failure and something is caught in Rainforest, I highlight that. For example, we use a “Caught in Rainforest” label in JIRA to measure the amount of tickets produced by Rainforest tests.
In addition to making sure that the developers understand the value of Rainforest, we’ve made an effort to involve them in the conversation about scaling Rainforest into the dev workflow. Rainforest’s CTO Russ came in to speak with one of our our lead developers about integrating Rainforest with Github and writing the tests in markdown [with Rainforest DevEx]. We were able to show our developers that Rainforest can be integrated into their workflow really easily.
What was one thing you wish you knew when you started using Rainforest?
I wish I had known not to get too granular with our tests. We initially spent a lot of time building out a full regression test suite and running all those tests. What we found was that a really large regression test suite was challenging for a single person to maintain and also that we were using a large amount of steps for each run, when really our priority was on ensuring our critical path tests were green. Really focusing on testing the paths that matter the most for us would have been a more efficient strategy from the start.
What advice would you give to a new user about setting up Rainforest at their organization?
Focus on the critical path and getting those up and running. Make sure you’re never writing the same test twice. Break things into chunks as much as possible, use embedded tests and think about your tests in small, reusable units. That’s something we’re trying to implement now, in order to make tests easier for our developers to integrate into their workflow.
Learn more about what Megan and her team are working on by visiting the Curalate website. And don’t forget to check out our previous installment of the Rainforest Customer Spotlight series with Sammy Samperi from iOffice.