Automated Smoke Testing: Everything You Need to Know

In software testing, smoke tests are a small set of end-to-end tests that make sure the most essential functions of an application or website work properly. 

Many software teams run a smoke testing suite of 10 to 15 tests as a preliminary step before running an entire regression suite (which could have anywhere from 50 to 500+ tests) to make sure the app is stable enough to merit further testing.

While it’s possible to do manual smoke testing, automating your smoke testing suite has two major benefits:

  • Automated smoke tests are much faster. Once the test is created, all you have to do is click a button to run all your smoke tests simultaneously. You can get results in a few minutes instead of waiting a few hours for a team of testers to complete manual testing.

  • Automated smoke tests can be scheduled to run regularly in production to confirm critical functionalities are working properly. You’ll be notified immediately if a run fails so you can address major bugs before they affect too many (if any) users. 

In this guide, we’ll discuss in more detail the benefits of automated smoke testing and show you how to use our tool, Rainforest QA, to automate your smoke testing suite without having to write any code.

Sign up for Rainforest QA—an all-in-one no-code automated testing tool that helps you do more testing in less time. You can run up to five hours of no-code automated tests for free, every month. It’s only $5/hour after that. 

What Test Cases to Include in Your Automated Smoke Suite

Since the main purpose of smoke testing is to make sure the major functions of your software work the way you expect them to, you only need to include a few test cases in your smoke testing suite. 

Here are two examples of what a smoke test suite might look like:  

  • For an e-commerce company, the most common and essential user tasks would be search, add to cart, check-out, place order, etc. By testing these paths, you’re guaranteeing that users can always do the most important thing on your site: buy something.

  • For a B2B web app like Slack, you’d want to test things like signing up with a new email, creating a new workspace, creating a channel, and sending a direct message.

To decide what tests to include in your smoke testing suite, identify the five to ten most common and essential user paths. Then, create the shortest tests possible to verify those user paths are working correctly.

When to Do Smoke Testing

Most QA teams use smoke testing to catch major defects in the software at the earliest stage possible so they don't waste time and resources on further testing of broken software. Whether they’re doing regression testing, hardware testing, or integration testing, they’ll first make sure each software build can pass the smoke testing process before moving on to more exhaustive testing. 

Additionally, some teams use smoke testing as a stopgap when they don’t have the time or resources to run their entire regression suite as often as they’d like. If they can only run the full regression suite once a week, they might conduct smoke testing after submitting each piece of new code to catch any major defects soon after they’ve been created. When used for this purpose, teams may refer to smoke testing as “build verification testing” or “build acceptance testing”. 

Another way to use smoke testing is to schedule a few tests to run on your live site at regular intervals (such as every hour). This helps you make sure critical functions are always running properly and gives you the opportunity to fix issues right away. 

How to Automate Smoke Tests without Coding

Rainforest QA is an all-in-one no-code test automation platform that lets you scale your testing — including smoke testing — without hiring more testers. 

Here’s how Rainforest QA can help you do more testing in less time:

Write and Edit Tests without Any Coding

With Rainforest QA, you can create automated functional tests in a matter of minutes, even if you have no technical background.

To write a Rainforest test step, you choose from a drop-down menu of actions (such as “click” or “verify”), and click-and-drag the mouse to take a precise screenshot of the element you want to apply the action to. Then, you can preview each step individually or preview the test as a whole to verify that each step will do what you want it to do. 

Adding a Click Action in Rainforest QA

This is a much faster process than the traditional approach to automated testing, which involves writing lines of code for each step. 

For even faster test creation, Rainforest QA lets you embed one test in another test. For example, if you have a test for a signup flow, you can embed that test into every other test that starts with a signup flow. 

Rainforest QA Signup Flow

After you’ve written tests for all the test cases in your smoke suite, you can start a test run with just a few clicks. Your tests will run simultaneously on our network of virtual machines, returning results in less than four minutes, on average.

You can also use our CLI to run Rainforest tests in any software development environment. For custom workflows and more complex data analysis, our API makes it possible to integrate Rainforest into any software stack, helping you achieve a fully-automated continuous integration/continuous release (CI/CD) pipeline. 

Get Better Test Coverage by Mimicking Real Users

Most automated testing tools (such as Selenium, Cypress, or Katalon) are designed to automate actions inside a single browser tab. Rainforest takes a different approach. The tests you create with Rainforest automate a virtual machine to mimic exactly how a real user would interact with the page (e.g., by moving the mouse over elements they can see on the screen, clicking on elements, typing in forms, and observing images). 

This testing method provides two major benefits for any type of testing on the front-end, including smoke testing: 

  • You can be confident that the test will catch any visual bugs, since you’re testing what a real user would see. Most other testing tools require workarounds to catch visual bugs.

  • You can include actions that actually leave the browser — something you can’t do with an automation framework that can only manage tests inside of a single webpage — like downloading a desktop app, installing it, and opening it. If one of the first steps users need to take to use your application is to download a file or configure something outside of the browser, it’s important to be able to include that step in your smoke tests.

Quickly and Easily Maintain All Your Tests

Most tests eventually need to be updated to accommodate new functionalities and updates in the application. With Rainforest, you can update any test step in the same way it was created — by choosing from a list of preset actions and taking screenshots. 

Rainforest QA also has a built-in suggested fix feature that gets triggered any time a perfect (or nearly perfect) match to the original screenshot can’t be found. Whatever screenshot on the page is closest to the original screenshot will be suggested as an update. 

A preview of Rainforest QA's built-in suggested fix feature

Then, you can click ‘Update Screenshot’ to automatically update the test if the suggested screenshot is an acceptable change. You can also choose to manually take a new screenshot.

If you’re using embedded tests, when you make a change in one place, it will automatically be updated in every other test that uses that embedded test. 

Easily Understand Why Every Test Failed

One of the most time-consuming aspects of testing for many teams is figuring out why tests failed. 

Rainforest QA automatically records a video of every test, whether it passes or fails. You can compare each test run to quickly see if anything in the user interface changed between runs. Video replays also show you exactly how the failure would’ve appeared to a real user, which can help you decide how important it is to fix the failure.

When watching video replays, the test script will appear to the left of the video. Any step that fails is highlighted in red and can be looked at in greater detail by clicking the ‘investigate action’ button below it. 

Investigate Action Button in Rainforest QA

For every test result, Rainforest also provides HTTP logs and browser logs. These details are often the key to identifying failures caused by hiccups in the test environment.

Additionally, Rainforest QA offers a Jira integration so anyone assisting with the testing process can automatically create a ticket for the development team. The ticket includes the failed test steps, a screenshot of the failed test step, HTTP logs, and a link to the full test results and video recording in Rainforest. 

Get 5 Hours of Free Testing Each Month, Then Only Pay for What You Use

To begin using automation to test a web application, you need a way to write test cases, manage and organize the test suite, run tests, and understand why the tests failed. Many automation tools will charge you extra for each of these key features or won’t provide them at all. If you’re in the early stages of QA testing and just looking to get started with a small set of tests, like smoke tests, managing and paying for each feature can be a hassle.

Rainforest QA is an all-in-one platform, which means you’ll have access to everything you need to create, execute, and manage any number of test cases you need to achieve your goals for quality assurance. You can start for free with up to 5 hours of functional testing per month—which is enough for most teams to build a few smoke tests and run them a couple of times to test out the platform.

It’s only $5/hr after that and you only pay for what you use. That is, you only pay for the time spent running tests, not for previewing, writing, editing, or otherwise managing your test suite. So, if the test fails on step one because of a bug, you’ll only pay for a few seconds of run-time.

If you think Rainforest could be the right fit for your testing team, sign up today.

Related articles

How to Start Automation Testing from Scratch without Code

We provide practical guidance on how to start automation testing from scratch and how to choose a test automation tool.

Top 10 Tools for Automated Front-End Testing in 2022

10 automated front-end testing tools that can help you release new features faster with fewer bugs.

8 Best Practices to Reduce Test Automation Maintenance

With these test automation maintenance tips, prevent automated test breakage and spend less time fixing broken tests.

How to Automate Regression Testing So Anyone Can Do It

In this post, you'll learn how to automate regression testing without having to know any code.