{"id":378,"date":"2020-10-09T23:08:04","date_gmt":"2020-10-09T23:08:04","guid":{"rendered":"http:\/\/rainforestqa.com\/the-layers-of-testing-architecture\/"},"modified":"2025-03-05T23:02:04","modified_gmt":"2025-03-05T23:02:04","slug":"the-layers-of-testing-architecture","status":"publish","type":"post","link":"https:\/\/www.rainforestqa.com\/blog\/the-layers-of-testing-architecture","title":{"rendered":"The layers of testing architecture"},"content":{"rendered":"\n<p>The landscape of software testing is changing. In the hyper-competitive world of technology, speed and quality are often seen as opposing forces. We are told to \u201cmove fast and break things\u201d if we are to succeed in getting our products into the hands of users before our competition beats us to the punch. This often times means sacrificing quality and confidence in the name of getting new features out the door.<\/p>\n\n\n\n<p>While this trade-off sometimes makes sense, it inevitably comes back to haunt you in the form of technical debt, bugs, decreased user confidence, and blockers for your product and engineering teams. As those teams scale, we aim to ship faster and faster. Meanwhile, the frameworks and technologies we build on top of become more dynamic. Traditional approaches to test automation are not only falling short, but they are becoming a burden rather than an asset.<\/p>\n\n\n\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_82_2 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Contents<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/www.rainforestqa.com\/blog\/the-layers-of-testing-architecture\/#Testing_architecture\" >Testing architecture<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/www.rainforestqa.com\/blog\/the-layers-of-testing-architecture\/#Layer_1_unit_tests\" >Layer 1: unit tests<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/www.rainforestqa.com\/blog\/the-layers-of-testing-architecture\/#Layer_2_service_integration_tests\" >Layer 2: service (integration) tests<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/www.rainforestqa.com\/blog\/the-layers-of-testing-architecture\/#Layer_3_UI_tests\" >Layer 3: UI tests<\/a><\/li><\/ul><\/nav><\/div>\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Testing_architecture\"><\/span>Testing architecture<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>In order to understand where Rainforest automation fits into a companies testing strategy and the enormous benefits it brings, we first need to understand the basics of testing architecture.<\/p>\n\n\n\n<p>There are many layers to properly testing an application and the many services it interacts with. For simplicity, we will represent general testing architecture with an oversimplified model: The Test Pyramid.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/uploads-ssl.webflow.com\/60da68c37e5767dfb65004c0\/60da68c37e57671b0f5008b9_s_CC71C99CE8C6D222C15F7BBB78309350BEE201DC802D4918B87E5BD9FAF212A1_1591828972117_image.png\" alt=\"\"\/><\/figure>\n\n\n\n<p>This model originally comes from Mike Cohn\u2019s book &#8220;<a href=\"https:\/\/www.mountaingoatsoftware.com\/books\/succeeding-with-agile-software-development-using-scrum\" target=\"_blank\" rel=\"noopener\">Succeeding with Agile<\/a>\u201d and consists of three layers:<\/p>\n\n\n\n<ol start=\"1\" class=\"wp-block-list\">\n<li>Unit tests<\/li>\n\n\n\n<li>Service tests (a.k.a. integration tests)<\/li>\n\n\n\n<li>UI tests<\/li>\n<\/ol>\n\n\n\n<p>This model does not adequately capture the many layers of testing modern applications, but the mental model is ideal for generalizing and understanding the concepts of testing.<\/p>\n\n\n\n<p>At the bottom of the pyramid, tests are very granular and are executed quickly and at a very low cost (essentially free). As we move up the pyramid, tests become more generalized, slower, and more expensive (both in terms of execution <em>and<\/em> creation\/maintenance).<\/p>\n\n\n\n<p>Consider the following tech stack for our theoretical application:<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"666\" height=\"526\" src=\"https:\/\/www.rainforestqa.com\/blog\/wp-content\/uploads\/2020\/10\/tech-stack-diagram.png\" alt=\"\" class=\"wp-image-2818\" srcset=\"https:\/\/www.rainforestqa.com\/blog\/wp-content\/uploads\/2020\/10\/tech-stack-diagram.png 666w, https:\/\/www.rainforestqa.com\/blog\/wp-content\/uploads\/2020\/10\/tech-stack-diagram-300x237.png 300w\" sizes=\"(max-width: 666px) 100vw, 666px\" \/><\/figure>\n\n\n\n<p>In our example, the frontend (User Interface) is a React app which talks to our Backend API. That API is responsible for reading\/writing to the database, talking to our other microservices, and interacting with external third-party APIs.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Layer_1_unit_tests\"><\/span>Layer 1: unit tests<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>The bottom layer of the pyramid is the first line of defense against bugs. It\u2019s used to test specific pieces of our tech stack. In particular, it tests \u201cunits\u201d of each application\/service, which is a somewhat arbitrarily defined \u201cchunk\u201d of code. There are a lot of nuanced decisions to be made by your engineering team around what defines a \u201cunit\u201d and how they should be tested, but that\u2019s outside the scope of this article.<\/p>\n\n\n\n<p>Each red box in our tech stack is tested with its own set of unit tests.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"668\" height=\"528\" src=\"https:\/\/www.rainforestqa.com\/blog\/wp-content\/uploads\/2020\/10\/red-box-tech-stack-diagram.png\" alt=\"\" class=\"wp-image-2819\" srcset=\"https:\/\/www.rainforestqa.com\/blog\/wp-content\/uploads\/2020\/10\/red-box-tech-stack-diagram.png 668w, https:\/\/www.rainforestqa.com\/blog\/wp-content\/uploads\/2020\/10\/red-box-tech-stack-diagram-300x237.png 300w\" sizes=\"(max-width: 668px) 100vw, 668px\" \/><\/figure>\n\n\n\n<p>Unit tests are performed in complete isolation and in artificial or simulated environments. This means the React unit tests know nothing about our Backend API, and vice versa. We are testing the internal pieces of each application and making sure the individual parts work as intended. An example of this is testing a single React component &#8211; for our example, we\u2019ll use a simple button.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>const MyButton = ({ onClick, text }) =&gt; (\n  {text}\n);<\/code><\/pre>\n\n\n\n<p>Our unit tests will test two things:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>the button renders with the text that is provided to it<\/li>\n\n\n\n<li>when the button is clicked, it invokes the onClick provided function<\/li>\n<\/ul>\n\n\n\n<p>Using Jest + Enzyme (which are standard libraries for testing React applications), our unit test looks something like:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>describe('Button', () =&gt; {\n  beforeEach(() =&gt; {\n    props = {\n      text: 'hello world',\n      onClick: jest.fn()\n    };\n\n    component = mount();\n  });\n\n  it('renders the text', () =&gt; {\n    expect(component.text()).toEqual('hello world');\n  });\n\n  it('invokes onClick prop when clicked', () =&gt; {\n    component.simulate('click');\n    expect(props.onClick).toHaveBeenCalled();\n  });\n});<\/code><\/pre>\n\n\n\n<p>Fantastic, the tests passed and we are now confident that our React code works. This is the standard way of testing a simple component and there is nothing wrong with this, but there are important things to note:<\/p>\n\n\n\n<ol start=\"1\" class=\"wp-block-list\">\n<li>This test runs in complete isolation &#8211; it has zero context about where it will be used in the application.<\/li>\n\n\n\n<li>It makes a naive assumption that the onClick function being provided to it is valid. By \u201cvalid\u201d I mean that the onClick value is <em>actually a function<\/em> (I could easily provide a number by accident), and the function isn\u2019t riddled with bugs. In short, it assumes that the engineer will use MyButton correctly.<\/li>\n\n\n\n<li>We are writing code to test code.<\/li>\n<\/ol>\n\n\n\n<p>We can mitigate the risk of #1 with solutions like static type checking, but concern #2 is inherently vulnerable to bugs and false positives. Should you write code to test the code that tests your code? Every time we writing unit tests, we make the assumption that our test code is valid &#8211; which is not always true.<\/p>\n\n\n\n<p>This is not a huge problem though &#8211; remember that this is only layer 1. We cannot expect layer 1 to catch <em>all<\/em> the bugs. If we flip our pyramid upside and visualize it as a funnel, we can see how the possible number of bugs are reduced as we push the code through the layers<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"906\" height=\"784\" src=\"https:\/\/www.rainforestqa.com\/blog\/wp-content\/uploads\/2020\/10\/bugs-reduced-through-layers.png\" alt=\"\" class=\"wp-image-2821\" srcset=\"https:\/\/www.rainforestqa.com\/blog\/wp-content\/uploads\/2020\/10\/bugs-reduced-through-layers.png 906w, https:\/\/www.rainforestqa.com\/blog\/wp-content\/uploads\/2020\/10\/bugs-reduced-through-layers-300x260.png 300w, https:\/\/www.rainforestqa.com\/blog\/wp-content\/uploads\/2020\/10\/bugs-reduced-through-layers-768x665.png 768w\" sizes=\"(max-width: 906px) 100vw, 906px\" \/><\/figure>\n\n\n\n<p>Now that we are relatively confident that our code works on a granular level, we can move to the next layer of our test pyramid.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Layer_2_service_integration_tests\"><\/span>Layer 2: service (integration) tests<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>The terms \u201cservice\u201d and \u201cintegration\u201d are interchangeable, and they are both quite vague. Similar to our vague definition of \u201cunits\u201d in our unit tests, our \u201cintegration tests\u201d live at the boundaries of our applications and test its interactions with external services, such as databases and third-party APIs.<\/p>\n\n\n\n<p>In our visualization, we are testing the red connections between applications\/services.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"668\" height=\"526\" src=\"https:\/\/www.rainforestqa.com\/blog\/wp-content\/uploads\/2020\/10\/red-connections.png\" alt=\"\" class=\"wp-image-2822\" srcset=\"https:\/\/www.rainforestqa.com\/blog\/wp-content\/uploads\/2020\/10\/red-connections.png 668w, https:\/\/www.rainforestqa.com\/blog\/wp-content\/uploads\/2020\/10\/red-connections-300x236.png 300w\" sizes=\"(max-width: 668px) 100vw, 668px\" \/><\/figure>\n\n\n\n<p>Integration tests can not be run in an entirely isolated, simulated environment like our unit tests. They require some testing infrastructure to be set up so we can test the interactions between services.<\/p>\n\n\n\n<p>For example, we want to test that our Backend API properly interacts with our database. The execution of one of the tests might look something like:<\/p>\n\n\n\n<ol start=\"1\" class=\"wp-block-list\">\n<li>Start the database and API<\/li>\n\n\n\n<li>Connect the API to the database<\/li>\n\n\n\n<li>Make an HTTP to the API that requires it to write to the database<\/li>\n\n\n\n<li>Read from the database to ensure the expected data has been written<\/li>\n<\/ol>\n\n\n\n<p>Not only do we test the interaction between API &lt;=&gt; DB, but this can also be seen as a superset of our unit tests. If our integration tests pass, it\u2019s reasonable to assume the API\u2019s code is working as expected (which is what our unit tests are for). We get redundancy for free!&lt;\/=&gt;<\/p>\n\n\n\n<p>We can test our integration with third-party services in a similar way:<\/p>\n\n\n\n<ol start=\"1\" class=\"wp-block-list\">\n<li>Start the API<\/li>\n\n\n\n<li>Connect the API to the third-party service<\/li>\n\n\n\n<li>Trigger a function in our API that requires it to read from the third-party service<\/li>\n\n\n\n<li>Check that our API handles the response correctly<\/li>\n<\/ol>\n\n\n\n<p>The important things to note for our integration tests:<\/p>\n\n\n\n<ol start=\"1\" class=\"wp-block-list\">\n<li>We are no longer running tests in complete isolation. We need to setup a testing environment (a functional clone of our production application, minus some data and scaling capabilities) which has the added benefit of testing the infrastructure that runs our application. It\u2019s ideal for this environment to mirror our production environment as closely as possible.<\/li>\n\n\n\n<li>Although we are not in <em>complete<\/em> isolation anymore, we have still isolated specific pieces of our tech stack. It\u2019s possible that bugs could occur when multiple services interact with each other. In essence, we are just testing larger \u201cunits\u201d of our tech stack. We must control the state of the system, which is obviously necessary to execute the tests, but does not necessary emulate \u201creal world\u201d input that comes from users.<\/li>\n\n\n\n<li>We are still writing code to test our code. It\u2019s possible that our testing code could have bugs.<\/li>\n<\/ol>\n\n\n\n<p>There are many different sub-layers to our integration layer, and these become more complex as your tech stack grows. A complex <a href=\"https:\/\/blog.newrelic.com\/technology\/microservices-what-they-are-why-to-use-them\/\" target=\"_blank\" rel=\"noopener\">microservice architecture<\/a> can be great when you have many teams working in parallel, but you need to ensure that the integrations don\u2019t break. These risks are mitigated with approaches like <a href=\"https:\/\/pactflow.io\/blog\/what-is-contract-testing\/\" target=\"_blank\" rel=\"noopener\">contract testing<\/a>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Layer_3_UI_tests\"><\/span>Layer 3: UI tests<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Our final layer allows us to test our application in a \u201creal world\u201d scenario. We are no longer testing individual pieces &#8211; instead, we are testing our entire tech stack from the perspective of our end-user.<\/p>\n\n\n\n<p>An industry standard approach to UI testing is by writing automation scripts with Selenium. There are other technologies and solutions, but they ultimately take the same approach &#8211; DOM based testing. An exception to this would be using humans to manually do your QA, but for obvious reasons this it not as scalable as an automated approach.<\/p>\n\n\n\n<p>Our different applications\/services interact with each other, but our users ultimately interact with our services through a single entry point: the user interface. This interaction is shown as the red arrows below.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"666\" height=\"728\" src=\"https:\/\/www.rainforestqa.com\/blog\/wp-content\/uploads\/2020\/10\/ui-entry-point.png\" alt=\"\" class=\"wp-image-2823\" style=\"width:840px;height:auto\" srcset=\"https:\/\/www.rainforestqa.com\/blog\/wp-content\/uploads\/2020\/10\/ui-entry-point.png 666w, https:\/\/www.rainforestqa.com\/blog\/wp-content\/uploads\/2020\/10\/ui-entry-point-274x300.png 274w\" sizes=\"(max-width: 666px) 100vw, 666px\" \/><\/figure>\n\n\n\n<p>Due to the requirements of properly testing the interaction, we are forced to implicitly test our entire tech stack:<\/p>\n\n\n\n<ol start=\"1\" class=\"wp-block-list\">\n<li>An entire testing environment is required, which means running each of our applications\/services.<\/li>\n\n\n\n<li>Tests must be run in an actual web browser (or equivalent environment for things like desktop apps, mobile apps, etc).<\/li>\n\n\n\n<li>Tests must be executed <em>only<\/em> by interacting with the UI &#8211; just like a real user would &#8211; and checking that the UI is updated properly.<\/li>\n<\/ol>\n\n\n\n<p>It may not be immediately clear why this is implicitly testing our entire tech stack. Consider what happens when you click a button in the UI:<\/p>\n\n\n\n<p>In layers 1 &amp; 2, we tested each of these \u201cunits\u201d individually. Even though this is a simple button click, we\u2019ve tested that <em>many<\/em> pieces of our tech stack work in concert with each other.<\/p>\n\n\n\n<p>The industry standard approach to layer 3 is DOM-based testing with tools like Selenium and Cypress. However, Rainforest Automation is the future of UI testing.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The landscape of software testing is changing. Speed and quality are no longer seen as opposing forces.<\/p>\n","protected":false},"author":4,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"content-type":"","inline_featured_image":false,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-378","post","type-post","status-publish","format-standard","hentry","category-qa-strategy"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.rainforestqa.com\/blog\/wp-json\/wp\/v2\/posts\/378","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.rainforestqa.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.rainforestqa.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.rainforestqa.com\/blog\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.rainforestqa.com\/blog\/wp-json\/wp\/v2\/comments?post=378"}],"version-history":[{"count":6,"href":"https:\/\/www.rainforestqa.com\/blog\/wp-json\/wp\/v2\/posts\/378\/revisions"}],"predecessor-version":[{"id":2973,"href":"https:\/\/www.rainforestqa.com\/blog\/wp-json\/wp\/v2\/posts\/378\/revisions\/2973"}],"wp:attachment":[{"href":"https:\/\/www.rainforestqa.com\/blog\/wp-json\/wp\/v2\/media?parent=378"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.rainforestqa.com\/blog\/wp-json\/wp\/v2\/categories?post=378"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.rainforestqa.com\/blog\/wp-json\/wp\/v2\/tags?post=378"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}