Real world device testing is often touted as a good thing for mobile testing, and in some limited circumstances it might be. However, in the vast majority of cases, it's a waste of time – both for your testers and team.
Why? Results end up being unrepeatable without the original device, which has a constantly changing state. This is especially painful with a classic outsourced service, where testers are constantly installing new apps, loading different sites, installing extensions and updates. However, of course your team will try as all good teams do – to repeat the bug. This where the major time-suck comes in – how long do you attempt to replicate the bug for? When do you give up? Was the bug caused by a weird extension that the tester had installed, or because they'd rooted their phone? You'll never know…
At Rainforest, we exclusively use virtual machines, or real devices from Amazon Device Farm. Amazon Device Farm is a service from AWS that provides real devices in a clean state on demand – it’s the future of testing devices according to Amazon (and we believe them too). The combination allows us to have the right device available at the right time to any tester in a clean, known state (optionally configurable to your needs, of course). What this means for Rainforest customers is that each test on a site or app will start from a consistent, repeatable state every single time – this enables a modern QA process.
So, regardless of whether you're a tester, or a customer trying to replicate a bug, we've got you covered. Stop wasting your time and focus on getting fast, repeatable results now. Use something you’ll actually get applause from – try Rainforest QA today.