In the Rainforest Customer Spotlight series, we highlight the superstars of the Rainforest QA platform: the Rainforest project managers (PMs) that write tests, manage databases and help build better quality processes for their organizations.
In this week’s post we speak with Heath Hughes, Lead QA Engineer at Birch Grove Software, who has been using Rainforest since January 2017. Read on to find out how Heath uses Rainforest to offload testing and find bugs faster, and the impact that Rainforest has had on his product quality.
Our main product is ActivTrak. It’s a user behavior analytics app that lets organizations document and understand how work is completed and provides data to optimize efficiency and increase productivity. It consists of a web-based control center and tracking agents, which are installed on each machine to be monitored.
For example, if an employee is spending too much time in one place, administrators have the option to set an alarm, label the activity as unproductive, and block or terminate the website or application from being accessed. Basically, ActivTrak captures information to understand who’s doing what and for how long and provides the tools to actually change behavior and increase productivity across an organization.
I’m a QA engineer, and in September 2016 I was brought on as a QA engineer and tasked with managing testing and releases. Before I joined, we didn’t have much of an organized QA structure or process. Since then we added some additional testers, and I became the lead.
Our process has come a long way in the past six months. We went from an exclusively manual testing process to having a suite of over 180 test cases that we run every other week prior to release. Rainforest handles the bulk of our regression testing. As new features are added, we test them manually. Then we write Rainforest test cases to cover the most valuable scenarios.
We also have a small set of test cases that we have scheduled to run once a week to catch any major issues in the key flows of our application.
By freeing up the time I was spending on manual regression testing, Rainforest has allowed me to greatly expand my test coverage and get feedback faster. Over a third of the bugs found by Rainforest have been high priority issues, and our customer satisfaction ratings have continued to rise in the months since we started using Rainforest.
We use Bamboo for deployments, JIRA for issue tracking, PractiTest for managing test cases for new features, Slack for team communication and Rainforest results. I also use Protractor to automate test data creation. It comes in handy to create logins for the Rainforest tabular variables.
For the first 2-3 months, I spend about 75% of my time focused on building out our Rainforest test suite. That initial time investment allowed me to see a snowball effect of getting more of my time back as I offloaded more and more work to Rainforest.
After reporting and resolving bugs, my top priority is staying on top of Rainforest test case maintenance. If I’m up to date with that, I work on writing new test cases for new features, based on a backlog that I update every sprint. I spend at least 25-50% of my time in Rainforest, sometimes more.
Every other week, usually on Wednesday evening, I run the entire regression suite. The next day I review the results, then I push out the release when it’s ready.
Putting in the time and effort at the beginning to learn the platform, and building out our test suite so we could get started as soon as possible. Because we’re such a small team, it was crucial for me to get a high-quality test suite up and running quickly, to free up my time to work on other high-priority tasks.
On the business side, we experienced a little bit of skepticism around whether Rainforest would really be worth the investment, as opposed to adding more testers to the team.
Communicating the results we were seeing from Rainforest was important to overcoming that. I would show my team the bugs that Rainforest has brought to light. We’re seeing an average 1-2 bugs on average surfaced by Rainforest per run, and over a third of those are high priority bugs. Demonstrating the amount of coverage we have is powerful.
It took some time to learn how to write my tests to be as short and simple as possible. The tests I write now are much shorter, more focused and less flaky than the ones I wrote at the beginning.
Check out ActivTrak to learn more about what Heath and his team are building at Birch Grove Software. Catch up on our previous posts on Rainforest project managers here: