With an insane 17.2 billion app downloads in Q1 of 2016 alone, the mobile app industry is exploding. With so many new apps on the market, the bar for acceptable quality has shot up and developers have to think strategically about mobile app testing to stay ahead of the curve.
Recently, Rainforest CTO Russell Smith sat down with Trent Peterson, AWS Device Farm product manager, to talk about how Device Farm was built to help developers test mobile apps on real devices more efficiently. Here are three things we learned about mobile app testing during the webinar.
While mobile QA may seem like simply a matter of keeping a drawer full of relevant devices somewhere in your office, executing mobile testing at scale requires much more than that. Trent told Russ: “The root of all of this was just talking to developers and seeing these makeshift device labs. As engineers, we oftentimes will see a problem like, ‘Oh, we get to buy a bunch of devices and rig them up for automation.’ It sounds like a cool weekend project. The problem is that it quickly goes from a weekend project to, ‘We need to staff this,’ and so on.” Mobile QA costs more than just the expense of purchasing devices. A fully-fledged mobile app testing setup requires staffing and the infrastructure to maintain and integrate devices into QA testing workflows.
Trent co-founded AppThwack in 2012 to give developers access to Android devices online so they could easily run fuzz tests their apps. They realized that mobile app testing was a much more widespread problem than they initially anticipated. After joining AWS, his team created Device Farm and expanded their offering to include iOS devices and add a broader range of testing functionality.
Trent points out that fragmentation is about more than just a variety of devices and OS versions to consider. Carrier settings can impact app performance -- from pre-installed apps to background processes to disabled or modified behavioral settings. “When it comes to carriers, there’s different types of modifications that are done...Certain settings might be completely disabled,” says Trent. Environmental issues, like connectivity, device charge, and users running other applications simultaneously can also impact user experience in a meaningful way. The complexity of fragmentation means that testing mobile apps on real devices -- and not just emulators -- is a necessary step in ensuring quality.
Trend and Russ also discussed how QA teams are taking on a very different shape as mobile testing evolves. “We’re seeing is that there’s this evolution, especially in terms of mobile, where the QA team is more in charge of actually putting in place the infrastructure and tooling to make testing possible, but developers are actually writing and maintaining the tests,” says Trent. As developers take on a more hands-on role with test execution, there’s a greater need for scalable, streamlined mobile app testing. Russ pointed out that less technical team members -- such as product managers -- are also getting involved in QA strategy, which means that making mobile testing accessible to the whole team is also critical.
AWS Device Farm and the Rainforest platform provide development teams with a new way to test their mobile apps that doesn’t slow them down. To hear how Trent and his team built AWS Device Farm and how we’re using it to provide scalable mobile testing on the Rainforest platform,download the webinar and listen now!
In this post, we'll cover the most common types of web application testing, which ones to prioritize, and tools to use.
Learn how to write a software test plan. Get started with our practical guide and free template.
Many quality assurance metrics evaluate QA activities rather than results. These are the 5 most helpful QA metrics.
Learn the basics of agile regression testing and five strategies to streamline your software testing process.