As described in a recent post, Rainforest QA relies on virtual machines (VMs) to provide our testers with reliable and repeatable testing environments. While this provides a lot of convenience, it also brings its own set of challenges, especially in the security field. In this blog post, we are going to give a brief introduction to some of the key measures we have taken to secure the VMs and protect our customers.
Managing the VM infrastructure is not too different than managing an enterprise network full of desktop computers. The benefit of a VM stack over an enterprise network is the disposable nature of the VMs. We can spin up new VMs on demand and destroy them after being used at a negligible cost.
Once our automated provisioning system decides we need a VM of some type, it will initialize the correct VM and start adding the monitoring services. In this stage, we will create a clean VM. We also spin up monitoring services for things like screenshot and video recording at this point. This way we can analyze everything than happens inside the VM and reproduce its session.
After the VM has been initialized and the monitoring services are up and running, we generate a new, unique connection credentials for our remote desktop client to connect to it. Then we let the scheduler know it’s ready for being used.
When needed, a VM is assigned to a Job and credentials are transparently sent to our remote desktop client for a tester to use the VM. Only the assigned tester is able to access the VM.
Once the Job is done and the results are received, we destroy the VM. In order to ensure that the VMs are only used for testing purposes, VMs are destroyed after a set amount of time has passed, even if the Job is not completed.
During this process we make sure all monitoring information is saved and stored on a separate system and deprovision them. After this stage has happened, any information not taken out of the machine by our monitoring services is destroyed, and no one will be able to access it again.
As mentioned in the previous section, we take a lot of actions to make sure we can audit and reproduce everything that happened inside a VM. We do this not only for security reasons, but also for our customers to reproduce and debug issues that might get reported during testing.
Thanks to our transparent proxy we are able to record and make available all HTTP records that went out of the VM during the testing process. After recording the request the proxy sends them to a separate service to be processed, put together, compacted and stored in a secure and separate location.
Since the VM has been assigned to a tester for usage we start recording video of the tester session. At every step of a test, we also take a screenshot. As with the HTTP logs, those are saved separately from the VMs. So if there is any suspicious activity happening on any VM we can go back and see exactly how and when it happened.
Because we control the remote desktop connection with the VMs, we record all input devices actions. Every keystroke, mouse action and clipboard event is stored safely and fed into our AI engine for processing.
While having to maintain a network of desktop Virtual Machines on a self-owned network for 3rd parties to use sounds like a nightmare for security, we can take advantage of the nature of Virtual Machines to add stronger controls and audits than we could do with an enterprise network of bare metal machines, making the implementation of it less risky. Our control over our testing environments make this technology easier for both customers and testers to use, and a win from a security standpoint.
We hope you enjoyed this introduction to the Rainforest VM security! We’d love your feedback on how to make our processes even better.
Learn more about Rainforest virtual machine infrastructure in this introductory blog post.