Rainforest QA Customer Spotlight: Mark Anthony de Jesus from GrowthHackers
In this Customer Spotlight series, we highlight the superstars of the Rainforest QA platform: the Rainforest project managers that write tests, manage databases and help build better quality processes for their organizations.
This week we caught up with Mark Anthony de Jesus, a software engineer at GrowthHackers who has been using Rainforest QA since January 2017. Read on to learn how Mark Anthony’s team keeps their QA process as lean as possible.
- Profile: “Software Craftsperson”
- Time Allocated to Rainforest: 50% of time for first month; 1-2 hours weekly on an ongoing basis.
- Secret Sauce: Focus on designing test cases to be highly modular in order to create a highly efficient test suite with minimal redundancies. Approach the structure of your rainforest tests the same way you approach the structure of the code within your application. Advice: Leverage your Customer Success Manager’s experience to help implement Rainforest successfully.
What does GrowthHackers do?
GrowthHackers is a powerful collaboration platform helping marketers and growth professionals unleash their company's growth potential. We do this via a community of growth practitioners that share insights and resources at GrowthHackers.com, as well as a collaborative software platform, GrowthHackers Projects, that keeps teams on the same page as they accelerate their growth testing and learning process.
What’s your current role? What was your path to getting there?
My current role is an engineer, or “software craftsperson.” This is my second startup after coming from the world of big banks. I’ve been in several roles in the past, from engineer to manager to director, but with GrowthHackers, I was able to get back to my real love… Software development. Our team has been together for some time and has a lot of chemistry. I spend 90% of my time building and 10% pushing the QA process in our product.
Do you have an internal QA team?
We used to have an outsourced team, but in an effort to be more lean we made the decision to see if Rainforest could provide the same functionality and test coverage. We’re also in the process of developing a more robust testing process internally.
What does your QA testing process look like?
We start with code review -- whenever we file a pull request, we make sure that we have the right unit tests written. Our goal is that eventually every pull request will have Rainforest tests that sit side-by-side with the unit tests.
Every night we run a smoke test in our staging environment with Rainforest. Our Staging environment is not 100% like for like with our production specs, so we are using CircleCI to scale up our testing environments, reseed our test data, and kick off rainforest tests. We have it integrated with HipChat, so we’re able to see the results in the morning.
What does your average week as a Rainforest PM look like?
We spent a lot of time building out the test suite initially. For the first month, I spent about 50% of my day in Rainforest, and our lead QA person spent about 70% of her day in Rainforest. It was a big initial investment to move our manual tests into the Rainforest platform, but now it’s mostly small tweaks because everything is run automatically. Now we spend about an hour or two a week maintaining tests.
What impact has Rainforest QA had on your product quality?
The difference that Rainforest has made is that there has been no difference; we rolled off a dedicated outsourced QA team and replaced with with Rainforest, and there hasn’t been a hiccup. The fact that I’m not spending hours of my time managing this product means that I can spend my time building our product. After we made the initial investment in spinning up Rainforest, we haven’t had any interruptions to our coverage.
We’re also getting faster feedback; the full regression suite would have taken a couple hours for an in-house team. The feature that I love the most is the tester videos; I’m able to review what the Rainforest testers did and have the confidence that they ran the test correctly. We can replicate exactly what the tester did when they find an issue.
What was one thing you wish you knew when you started using Rainforest?
I wish I had fully understood that the testers have a varying levels of experience. People don’t always understand very technical or industry-specific terms. It’s important for us to be cognizant of writing our tests in layman’s terms. The more explicit we can make the tests, the fewer issues we’ve had with testers understanding what we were expecting of them.
What was your secret sauce that helped you ensure the success of Rainforest at GrowthHackers?
We’ve found that it’s important to be as focused and granular as possible when creating a test. For example, if we want to test the flow for creating a new experiment, we want to make sure that they’re just testing that flow, rather than trying to also test the signup flow within the same test case. We already have a test case for the signup flow, so including it in another test would be redundant.
It’s the same as when you code -- you don’t want your files to be thousands and thousands of lines if they don’t need to, because the bigger and more complicated it is, the harder it will be to debug. I follow that same logic with Rainforest test writing. If a test is really big and has 20 - 30 steps, we ask ourselves what we’re trying to accomplish with the test and zero in on a specific goal.
What advice would you give to a new user about setting up Rainforest at their organization?
Listen to the advice of your Customer Success Manager! They know what to do to set new customers up for success. Start small with a smoke test suite and then grow out your regressions. You’ll understand how Rainforest works more quickly and expand your test suite easier.
Learn more about what Mark Anthony and his team are working on by visiting GrowthHackers.com. And don’t forget to check out our previous installments of the Rainforest Customer Spotlight series with Sammy Samperi from iOffice and Megan Bruce from Curalate.