Rainforest Customer Spotlight: Heath Hughes from Birch Grove Software
In the Rainforest Customer Spotlight series, we highlight the superstars of the Rainforest QA platform: the Rainforest project managers (PMs) that write tests, manage databases and help build better quality processes for their organizations.
In this week’s post we speak with Heath Hughes, Lead QA Engineer at Birch Grove Software, who has been using Rainforest since January 2017. Read on to find out how Heath uses Rainforest to offload testing and find bugs faster, and the impact that Rainforest has had on his product quality.
- Profile: Lead QA Engineer
- Time Allocated to Rainforest: 75% of work time for first 2-3 months; 25-50% of time managing, writing and reviewing test cases after that.
- Secret Sauce: Investing time and effort to learning how to write clear, concise test cases at the beginning to start seeing results as quickly as possible.
- Advice: Leverage Rainforest Exploratory testers to create test cases to get a test suite up and running faster, and use features like tabular variables and embedded tests to shorten test run time.
What does your company do?
Our main product is ActivTrak. It’s a user behavior analytics app that lets organizations document and understand how work is completed and provides data to optimize efficiency and increase productivity. It consists of a web-based control center and tracking agents, which are installed on each machine to be monitored.
For example, if an employee is spending too much time in one place, administrators have the option to set an alarm, label the activity as unproductive, and block or terminate the website or application from being accessed. Basically, ActivTrak captures information to understand who’s doing what and for how long and provides the tools to actually change behavior and increase productivity across an organization.
What’s your current role? What was your path to getting there?
I’m a QA engineer, and in September 2016 I was brought on as a QA engineer and tasked with managing testing and releases. Before I joined, we didn’t have much of an organized QA structure or process. Since then we added some additional testers, and I became the lead.
What does your QA testing process look like?
Our process has come a long way in the past six months. We went from an exclusively manual testing process to having a suite of over 180 test cases that we run every other week prior to release. Rainforest handles the bulk of our regression testing. As new features are added, we test them manually. Then we write Rainforest test cases to cover the most valuable scenarios.
We also have a small set of test cases that we have scheduled to run once a week to catch any major issues in the key flows of our application.
How has Rainforest impacted your product quality?
By freeing up the time I was spending on manual regression testing, Rainforest has allowed me to greatly expand my test coverage and get feedback faster. Over a third of the bugs found by Rainforest have been high priority issues, and our customer satisfaction ratings have continued to rise in the months since we started using Rainforest.
What other apps, software, or tools are critical to your QA process?
We use Bamboo for deployments, JIRA for issue tracking, PractiTest for managing test cases for new features, Slack for team communication and Rainforest results. I also use Protractor to automate test data creation. It comes in handy to create logins for the Rainforest tabular variables.
How much time did you allocate for Rainforest during the initial onboarding?
For the first 2-3 months, I spend about 75% of my time focused on building out our Rainforest test suite. That initial time investment allowed me to see a snowball effect of getting more of my time back as I offloaded more and more work to Rainforest.
What does an average day (or week) in your role as a Rainforest PM look like?
After reporting and resolving bugs, my top priority is staying on top of Rainforest test case maintenance. If I’m up to date with that, I work on writing new test cases for new features, based on a backlog that I update every sprint. I spend at least 25-50% of my time in Rainforest, sometimes more.
Every other week, usually on Wednesday evening, I run the entire regression suite. The next day I review the results, then I push out the release when it’s ready.
What was your secret sauce that helped you ensure the success of Rainforest at your organization?
Putting in the time and effort at the beginning to learn the platform, and building out our test suite so we could get started as soon as possible. Because we’re such a small team, it was crucial for me to get a high-quality test suite up and running quickly, to free up my time to work on other high-priority tasks.
How did you prove the value of Rainforest internally? What are the metrics you use to show your success?
On the business side, we experienced a little bit of skepticism around whether Rainforest would really be worth the investment, as opposed to adding more testers to the team.
Communicating the results we were seeing from Rainforest was important to overcoming that. I would show my team the bugs that Rainforest has brought to light. We’re seeing an average 1-2 bugs on average surfaced by Rainforest per run, and over a third of those are high priority bugs. Demonstrating the amount of coverage we have is powerful.
What was one thing you wish you knew when you started using Rainforest?
It took some time to learn how to write my tests to be as short and simple as possible. The tests I write now are much shorter, more focused and less flaky than the ones I wrote at the beginning.
What advice would you give to someone who is currently setting up Rainforest at their organization?
- Keep your tests as short and simple as possible, without compromising their value. Break larger things into smaller, shorter tests and focus on testing one thing per test.
- Use Rainforest Exploratory testers at the beginning to help get your test suite built faster.
- Minimize extra steps by setting up as much of your data as possible outside of Rainforest and use tabular variables.
- Create embedded steps for common steps like navigating to each page and logging in to make sure each test really counts.
Check out ActivTrak to learn more about what Heath and his team are building at Birch Grove Software. Catch up on our previous posts on Rainforest project managers here: