Back to all case studies
The situation: Flux initially used Cypress for automated testing, but its complexity slowed down test creation and maintenance. Developers resisted using it, resulting in unreliable tests and escaped bugs.
The result: Rainforest's no-code interface makes test creation easy for Flux developers, leading to wider adoption and more test coverage. They spend 50% less time on manual QA and have 70% fewer customer-reported bugs.
Rainforest helped us to ship features faster by freeing up developer time to focus on actual feature work and not on regression and QA work.
With Flux’s web-based, real-time collaboration tool for electronics design and engineering, hardware development happens faster and better.
From the founding of Flux in 2019, CTO Christian Blank and his cofounders knew they wanted to develop code in a continuous integration / continuous development (CI/CD) pipeline — a pipeline that’d allow them to deliver new features and updates to their customers quickly and frequently.
But they also knew pushing code out fast wouldn’t help their customers or the company if the code was faulty. So, they first assigned their developers responsibility for code quality:
“A carpenter would never build a desk with drawers and then be like, ‘I don't even open the drawers because I just let the QA person see if it works.’"
Then they looked into automated solutions for end-to-end UI testing that’d allow them to move fast while ideally preventing defects from escaping to production.
Initially, they landed on a code-based testing solution, Cypress. Christian and his team quickly discovered that while Cypress was a good fit for certain use cases, it was a less-than-ideal fit for a lot of their end-to-end testing needs: Cypress offered flexibility, but with flexibility came complexity. This complex, specialized tooling made it take longer to write tests and more difficult to maintain tests, particularly because those tasks required engineers familiar with both Cypress and JavaScript.
Developers resisted using it, which resulted in insufficient and unreliable test coverage. A painful number of bugs were reported by customers. As Christian put it: “We got so frustrated with the bugs.”
So Flux’s developers spent more time doing manual QA to make sure everything in the product worked properly. All that time spent doing manual QA and fixing escaped bugs meant less time focusing on feature development.
Christian and the team decided that they wanted an automated testing solution that’d make it easy to create and maintain tests — both because they wanted developers to be inclined to adopt the solution, and because they wanted non-developers to be able to contribute to quality assurance: “For example, if we hire a hardware engineer to create content for us and they found a bug, we want them to be able to quickly create a test."
Christian considered a few different options, including Rainforest’s no-code test automation solution.
During the trial of Rainforest, Christian found that its intuitive, no-code approach did, indeed, make test creation and maintenance simple enough to incline the developers to use it.
“It’s a more natural process for people. They're way faster at adopting it, they’re way faster at creating tests. And that's a huge advantage of Rainforest for us.”
And, importantly, Rainforest noticeably prevented defects and regressions from escaping to production. Bug reports from customers dropped.
“We’ve definitely had less bugs since we started using Rainforest.”
Between the ease-of-use and positive impact on bugs, Rainforest found a foothold on the developer team. Today, everyone on the team is using Rainforest to catch bugs and other issues, while a majority of the team is involved in creating and maintaining tests.
Since it’s easy to quickly build and maintain end-to-end tests with Rainforest, Flux’s developers have gotten better at maintaining up-to-date automated test coverage.
They’ve dramatically improved product quality, even without the help of a dedicated QA team: Flux customers have reported 70% fewer bugs compared to the time before Rainforest.
The development team is also spending at least 50% less time doing manual QA. Now the time that the developers spend manually dogfooding the product is focused on optimizing the user experience, rather than confirming the product works as intended.
All of this has added up to less time spent on QA and more time shipping features — pushing code to prod upwards of 15 times per day.
