The Top 5 Test Writing Mistakes in Rainforest

Picture of Akhila Iruku
Akhila Iruku, Monday May 2, 2016

We’ve had many questions on our new tester training model and how to write a good test in Rainforest. To help our customers get the most out of Rainforest, our team has reviewed hundreds of Rainforest tests in the last few weeks to see how people have been writing tests. Our goal was to better understand why Rainforest users might get false positives, low NPS scores, or have a high test abandonment rate.

Our findings have been very helpful to improve how we train new customers to write tests. We wanted to share the five most common mistakes we noticed to help our customers write better tests in Rainforest for themselves.

5 Commons Mistakes in Test-Writing

Mistake 1 - Tests have too many steps.

If a Rainforest test is more than 10 steps long, this typically means that the test is no longer focused on one goal, but several goals.

What You Can Do: Review the test to see if there is any way it can be broken up or separated into embedded tests.

Mistake 2 - The step contains quoted text that is not an exact match for your site.

Testers are trained to match text in quotes exactly. ANY difference in spelling, content, capitalization, or punctuation will fail with a note explaining the problem.

What You Can Do: If you do not need your text to match exactly, avoid using quotation marks in your test steps.

Example

| This step without quotations will fail | This step with quotations will pass | This step without quotations will pass | |-------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------| | Click the "Log-in" button and fill out the form. Were you able to Login? | Click the "Login" button and enter first name: "{{random.firstname}}" and last name: "{{random.lastname}}". Were you able to Login? | Click the Log-in button and fill out the form. Were you able to Login? |

Mistake 3 - The step includes too many actions.

Writing a test with several unrelated actions means that you are asking a tester to go through several flows. If one of the action breaks, it may be unclear to you which part is not functioning in the result.

What You Can Do: Break out each action into separate steps. Each step should have only a single action.

Example:

Step with too many actions:

Correctly written test with actions broken out into multiple steps:

Mistake 4 - The step asks too many questions.

Asking a tester multiple questions can make it hard to pinpoint a bug/error. Testers are asked to provide a "Yes" or "No" answer to your question. When you ask multiple questions, it's possible they say "No" to one question and not the other. This makes finding the error more difficult for you.

What You Can Do: If you find yourself asking more than one question per step, break that step out into additional steps.

Incorrect Test with too many questions:

Correctly written test with questions broken out by steps:

Mistake 5 - The steps are too ambiguous.

There are many factors that can make steps challenging for testers to complete. For example: Are you asking a tester to remember or memorize information? Are you using internal jargon or terminology that a tester may not be familiar with? Navigating your site may not be as intuitive to a tester-- are your instructions clear when navigating to a specific element on the page?

What You Can Do: Take action to make your tests as clear and easy to complete as possible, including: - Use step variables to help testers fill out forms and account information that may be difficult to memorize or remember. - Check the terminology used in your test to ensure that it is easy to understand. - Be as precise as possible when directing a tester to find key elements on the page

Example

Incorrect test with too much ambiguous information:

Correctly written test with clear instructions, no company-specific terminology, using step variables:

If you have any questions, please reach out to csm@rainforestapp.com. Our customer success team can help you review tests together!