Manually testing an application is time-consuming, costly, and difficult to scale as your application grows: as you add more features to your application, you have to add more functional tests. And getting those additional tests done usually means adding headcount.
Automated functional testing can speed up the testing process, provide more consistent results, and give one person the ability to manage the testing workload of five or more manual testers.
But automated functional testing comes with its own challenges:
In this guide to automated functional testing, we’ll cover everything you need to know to get started and avoid as many of these challenges as possible. We’ll discuss:
Ready to automate your functional testing? Sign up for Rainforest QA—you can run up to five hours of no-code automated tests for free, every month. It’s only $5/hour after that.
Functional testing is the process of confirming that all the elements of an application work together as a whole and appear correctly in the user interface (UI) so that a user can perform desired actions.
Different types of functional testing include:
Unit testing is sometimes considered functional testing because it evaluates whether a unit of code behaves as intended. However, for the purposes of this article, we won’t include it as a type of functional testing because it doesn’t touch the user interface.
For example, say you’re building a calculator app. With unit testing, you can confirm the accuracy of the “sum” function. When given the input of “2+2,” the function returns the answer “4.” But to be fully confident that the calculator will work for users, you need to test how it works from the UI.
That’s where functional testing comes in. Functional testing would verify that the correct numbers and signs are actually appearing in the UI. Even if the “sum” function is working (i.e. the unit tests passed), a mistake in integrating other functionalities could cause a ‘2’ to appear on the visual layer when the backend of the app sends a ‘3’, causing the function to return the answer ‘5.’
Non-functional testing evaluates the app’s performance in terms of speed, security, and reliability. For example, a functional test would verify that a user can successfully complete a signup process. A non-functional test might test how long the sign-up process would take if 3,000 people were trying to access the web application at the same time.
A few types of tests that fall under non-functional testing include:
Before automated software testing, the only way to test your application was manually. It was common practice to jot down a few notes about the features that needed to be tested or create a spreadsheet of user paths (sometimes called data-driven testing). Then, someone would spend a considerable amount of time clicking through the application looking for bugs.
Even with the many test automation tools available today, a lot of teams still start with manual testing because it can be done without any additional training by whoever is available. Manual testing is a good option for test cases that frequently change or that require subjective judgment (“Do the images look clear?”). However, it’s not a long term, scalable solution for all your functional testing.
Teams run into these difficulties with manual testing:
Eventually, teams usually find they need more test coverage than is feasible to build and maintain with manual testing. That’s where automated testing comes in.
In 2004, Selenium was introduced as the first open-source web browser automation tool for software testing.
Selenium is a testing framework that helps developers write lines of code to make a browser mimic end-user interactions with a web page or application. The Selenium code searches for specific lines of code (i.e. locator IDs) that represent the presence of an element (such as a button) in the user interface. If the locator ID is found, the test assumes the element is visible and the test passes.
As Selenium evolved, developers were able to run tests in parallel, author tests for any scripting language (such as Java or Python), use record-and-playback to write simple tests with Selenium IDE, and perform cross-browser testing with the Selenium Webdriver. This changed the face of software testing because suddenly:
As use of Selenium became more common, other tools appeared that made it easier to write and organize automated test scripts and to run even more tests simultaneously. Some of the tools also made it easier to understand why tests failed by capturing screenshots of the UI or snapshots of the underlying code at the point of the failure.
But even with these improvements and add-ons, doing automated functional testing with Selenium and its offshoots still presents several challenges:
In response to this, commercial tools started appearing that offered “no-code” testing solutions. However, most of these tools simply generate the code for you. You will still run into the problems of:
Further, all testing solutions that test the underlying code—without interacting with the visual layer—may end up missing a lot of bugs that the majority of users would notice immediately.
The only way to get around these difficulties is to stop using code to test code.
Rainforest QA is a no-code automated functional testing tool that mimics real user behavior by interacting with the UI on the visual layer rather than with the underlying code. This means:
The next section takes a detailed look at how Rainforest QA differs from all other tools that test the underlying code.
Rainforest QA uses pixel-matching to locate elements and verify all aspects of the element (i.e. color, text, shape, position, etc.). With code-based tools, you would need to write a separate test step (or line of code) for each aspect of the element. Rainforest tests can verify all of that with just one screenshot.
To write or maintain a test step in Rainforest QA:
You can also choose to embed one test into another test to speed up test creation (and test maintenance).
For example, if you have a test for a signup flow, you can embed that test into every other test that starts with a signup flow.
Once you’ve written or embedded every test step, you can playback the test to verify it will run as intended. Rainforest QA also offers an API, CLI, and a CircleCI Orb so developers can kick off a suite of Rainforest tests along with their unit tests and integration tests.
Finally, Rainforest allows for multiple tests to be executed simultaneously on our cloud of virtual machines. You can run tests on the latest and older versions of Chrome, Firefox, Safari, and Microsoft Edge.
All code-based automation tools look for locators in the underlying code to verify if an element is visible. For example, let's say you want to verify that a message saying ‘signup was successful’ appears at the end of a signup process. Code-based tests will search through the underlying code for the element locator of the message. If the locator is present, the test will pass.
The problem with this is that even the slightest change in the element locator can break your test—even if the end results look identical to the user. Maybe your team renamed the locator during an update, or accidentally missed a letter in the locator when creating the test. Either way, a code based test would fail because it couldn’t find an exact match, even though the visual element would look and act the same way for the user.
Because Rainforest tests use pixel-matching to verify elements, they are less susceptible to minor changes in the underlying code. In fact, a Rainforest test will only fail if there is a visual change in the UI—which is more likely to represent a real bug.
Most automation solutions can only generate scripts that stay inside a single browser tab. With Rainforest’s proprietary automation, you can create tests that open new tabs and new windows, open and close browsers, and even interact with desktop apps.
For example, the video walkthrough shown above covers how to create a test that saves a file to the desktop from one location, and then uploads the file to Google Drive.
This feature helps test how your app interacts with desktop apps, how it interacts with browsers with ad-blockers, and other complex functions.
Rainforest records a video of every test (whether the test passes or fails) to make it easier to debug issues. Test results also include HTTP logs and JSON browser logs to help with debugging.
Rainforest QA also integrates with tools like Slack and Microsoft Teams, so you can get instant notifications for any test failure. If you integrate with Jira, every time you need a bug to be fixed, you can automatically create a ticket for the software development team that includes a video recording of the test and relevant logs.
This dramatically reduces the amount of time it takes to triage bugs.
Our free-forever Professional plan makes software test automation accessible to anyone. This plan has everything you need for test automation—parallel testing on virtual machines and unlimited team members—without any hidden costs. You get up to 5 hours of free testing every month and it's only $5/hr after that.
The most important factor to consider when adding automated functional testing to your release process is the stability of your product.
If you have a consistent series of functional tests that don't change very often and get run frequently, it’s likely that automation could save you a lot of time and money.
However, if your product or feature is still early-on in the development cycle, with major code changes occurring on a regular basis, it’s not time to automate yet. You’re better off waiting until the application is more stable.
Some test cases will never be a good fit for automation, which is why the Rainforest platform also offers on-demand manual testing by testers in our worldwide community of QA specialists. With one platform, you can scale up manual and automated testing without adding headcount.
Rainforest automation enables your team to move faster—without breaking things. It’s easy for teams to get started with automated UI testing and build as you go in an agile environment.
You don’t need to learn a new programming language, you don’t need to buy additional services from other vendors to run or manage your tests, and anyone on your team can create, run, maintain, debug, and interpret tests.
It’s a fast and scalable all-in-one solution that’s appropriate for teams that are just dipping their toes into automated testing, as well as QA-mature teams doing continuous testing with a suite of 500+ software tests.
Get started with Rainforest QA for free. You can run up to five hours of no-code automated tests for free, every month. It’s only $5/hour after that.
We provide practical guidance on how to start automation testing from scratch and how to choose a test automation tool.
10 automated front-end testing tools that can help you release new features faster with fewer bugs.
With these test automation maintenance tips, prevent automated test breakage and spend less time fixing broken tests.
In this post, you'll learn how to automate regression testing without having to know any code.