In this Customer Spotlight series, we highlight the superstars of the Rainforest QA platform: the Rainforest project managers that write tests, manage databases and help build better quality processes for their organizations.
This week we caught up with Mark Anthony de Jesus, a software engineer at GrowthHackers who has been using Rainforest QA since January 2017. Read on to learn how Mark Anthony’s team keeps their QA process as lean as possible.
GrowthHackers is a powerful collaboration platform helping marketers and growth professionals unleash their company's growth potential. We do this via a community of growth practitioners that share insights and resources at GrowthHackers.com, as well as a collaborative software platform, GrowthHackers Projects, that keeps teams on the same page as they accelerate their growth testing and learning process.
My current role is an engineer, or “software craftsperson.” This is my second startup after coming from the world of big banks. I’ve been in several roles in the past, from engineer to manager to director, but with GrowthHackers, I was able to get back to my real love… Software development. Our team has been together for some time and has a lot of chemistry. I spend 90% of my time building and 10% pushing the QA process in our product.
We used to have an outsourced team, but in an effort to be more lean we made the decision to see if Rainforest could provide the same functionality and test coverage. We’re also in the process of developing a more robust testing process internally.
We start with code review -- whenever we file a pull request, we make sure that we have the right unit tests written. Our goal is that eventually every pull request will have Rainforest tests that sit side-by-side with the unit tests.
Every night we run a smoke test in our staging environment with Rainforest. Our Staging environment is not 100% like for like with our production specs, so we are using CircleCI to scale up our testing environments, reseed our test data, and kick off rainforest tests. We have it integrated with HipChat, so we’re able to see the results in the morning.
We spent a lot of time building out the test suite initially. For the first month, I spent about 50% of my day in Rainforest, and our lead QA person spent about 70% of her day in Rainforest. It was a big initial investment to move our manual tests into the Rainforest platform, but now it’s mostly small tweaks because everything is run automatically. Now we spend about an hour or two a week maintaining tests.
The difference that Rainforest has made is that there has been no difference; we rolled off a dedicated outsourced QA team and replaced with with Rainforest, and there hasn’t been a hiccup. The fact that I’m not spending hours of my time managing this product means that I can spend my time building our product. After we made the initial investment in spinning up Rainforest, we haven’t had any interruptions to our coverage.
We’re also getting faster feedback; the full regression suite would have taken a couple hours for an in-house team. The feature that I love the most is the tester videos; I’m able to review what the Rainforest testers did and have the confidence that they ran the test correctly. We can replicate exactly what the tester did when they find an issue.
I wish I had fully understood that the testers have a varying levels of experience. People don’t always understand very technical or industry-specific terms. It’s important for us to be cognizant of writing our tests in layman’s terms. The more explicit we can make the tests, the fewer issues we’ve had with testers understanding what we were expecting of them.
We’ve found that it’s important to be as focused and granular as possible when creating a test. For example, if we want to test the flow for creating a new experiment, we want to make sure that they’re just testing that flow, rather than trying to also test the signup flow within the same test case. We already have a test case for the signup flow, so including it in another test would be redundant.
It’s the same as when you code -- you don’t want your files to be thousands and thousands of lines if they don’t need to, because the bigger and more complicated it is, the harder it will be to debug. I follow that same logic with Rainforest test writing. If a test is really big and has 20 - 30 steps, we ask ourselves what we’re trying to accomplish with the test and zero in on a specific goal.
Listen to the advice of your Customer Success Manager! They know what to do to set new customers up for success. Start small with a smoke test suite and then grow out your regressions. You’ll understand how Rainforest works more quickly and expand your test suite easier.
Learn more about what Mark Anthony and his team are working on by visiting GrowthHackers.com. And don’t forget to check out our previous installments of the Rainforest Customer Spotlight series with Sammy Samperi from iOffice and Megan Bruce from Curalate.