In the Rainforest Customer Spotlight series, we highlight the superstars of the Rainforest QA platform: the Rainforest project managers (PMs) that write tests, manage databases and help build better quality processes for their organizations.
For this post, we interviewed Ibrahim Taveras, QA Engineer for delivery.com. Read on to learn how Ibrahim has used Rainforest to increase test coverage, bring the development team closer to QA and cut regression testing time from hours to minutes.
At delivery.com, we enable customers to order online from local restaurants, wine and spirit shops, grocery stores, and schedule services with laundry and dry cleaning providers that deliver.
I started as a Junior QA Engineer, which entailed regression testing and manual testing for new features. Now I’m a QA Engineer, and I oversee our QA process from start to finish, from planning and testing to working with devs to release code to production.
Our process begins with planning. We prioritize which features we want to test for each sprint. The first portion of that entails feature QA. We go into a dev environment and test specifically the code that was changed for a new feature that’s going to be released. Once that’s passed in the dev environment, we create a release branch with other tickets and start regression testing on Rainforest, reducing the need for a manual regression by QA. This frees up more time, allowing QA to focus on new feature testing, enabling us to get new features to production faster.
If we find a bug in the dev environment, we reassign the issues to the developer. But if we find issues while we’re regression testing, the release managers will get the appropriate developers involved to push fixes. Once they redeploy, we restart regression testing.
We also take advantage of the organizational functionality in Rainforest, including tags and feature groups, which enable us to create customized test suites for smoke and regression tests. We can save time by knowing right away whether core functionality is broken or if we can release code.
Besides Rainforest, we utilize Google Spread Sheets to store our test cases and run manual testing against those cases. We also were using Selenium Web Driver for automation testing against our responsive sites which ran smoke tests against our core flow per every deploy using Jenkins integrations. We have transitioned to using Nightwatch.js to run our automated tests. In addition to Nightwatch.js for our responsive site, we have been setting up tests in Appium for automation testing on our native iOS and Android apps. Also, when running scripts in our dev environments, QA has been using Frisby.js with jasmine-node to set up users thus saving time from manually creating users.
We use the Rainforest integration with JIRA, which has been really helpful and saves a lot of time in terms of creating issues. I like that we have the ability to see video replay of test runs as well as network requests and HTTP headers. That saves our developers a lot of time when fixing issues.
At this point, I spend somewhere between 5-8 hours each week in Rainforest maintaining tests or adding new tests to our core regression flow. If I’m working on a new feature and I want to include it in our regression suite, I create a new test run for that specific feature or I add 3-4 extra tests to an existing test run to include the new feature.
The biggest difference that Rainforest has made is the amount of time we spend running regression tests on release days. Prior to the implementation of Rainforest, we spent a few hours regression testing to find bugs and redeploy after the issues were fixed only to test again, going back and forth until the code was in a state where we could get ready for a release to production.
We went from spending 3-4 hours running regression tests on one server to spending about 45 minutes receiving Rainforest test results. Rainforest has made a huge difference in terms of cost-efficiency and saving time.
Another benefit is test coverage. Without Rainforest, we had to be very selective about which tests we had time to run. Now we can include more tests without sacrificing time. If we have new features that weren’t regression tested before, but need to be included in every release moving forward, we can add a test run in Rainforest without increasing the time it takes to run our overall test suite.
At the very beginning we dedicated at least 2 weeks to get many of the initial tests set up. One of the biggest things I had to learn was how to be more detail-oriented in test-writing. To get to a comfortable place where the tests we write are clear and understandable to the testers, it took about another week -- maybe 10 additional hours of work.
About 2 months after rolling out Rainforest, we did a “Lunch and Learn” with our developers. We gave them a high-level overview of what Rainforest is and how we implemented it into our QA process. Ever since then, our developers have been on board and liking the Rainforest process.
Recently, we’ve been getting the devs involved in creating the tests, which improves the QA process but also helps devs stay involved in quality. As they’re coding a new project, they’ll be the first ones to start thinking about test coverage, and when it gets to QA we can refine those tests to make sure the feature is fully covered.
My biggest learning was to take advantage of the embedded tests and tabular variables. By setting up these variables you can implement them and write tests more quickly.
Additionally, think from the perspective of a user. You’re not writing these tests for yourself or your company, but for testers who are running the tests as a proxy for the users.
Read more user stories here.