Telecom Lead India: Nitin Dang, country general manager, India & SAARC, Micro Focus, says mobile test automation requires different methods to accommodate the near-constant change to platforms and architectures.
Mobility is the name of the game in the current landscape, the demand for mobile devices is huge and isn’t going away. Companies must, and will, increasingly engage with their customers ‘on the go’, through mobile apps The demand for smartphones and innovative applications is soaring and making the world of testing competitive and complex with each passing day.. A 100 million people will have smart phones and soon, there will be more linked devices on the planet than there will be people to use them! With the creativity on a high, there is increasing pressure on the developers and testers to build and test these mobile applications. There is a struggle to keep up without a change in thinking when it comes to mobile quality, performance and development.
Consumerization of IT through ‘Bring Your Own Device’ is expected to have the most significant impact on an organizations’ IT strategy in the next couple of years. As a Tester, you have to be prepared to deliver mobile apps that work first time, every time, on any device. Last year alone, there were 130 different Android devices running on seven different platforms and two firmware sets. This equates to an excess of 1,800 device combinations that should be tested to ensure business functionality is delivered to the market.
These days, smart phones and other mobile devices don’t simply carry the news–they make it. So interchangeable are these devices–at least on the surface–that the technology press now focuses on what each new device doesn’t have and cannot do rather than wowing us with what it can. This competition between device vendors means more people than ever have inexpensive access to powerful mobile computing platforms. For consumers, this means choice. For organisations building mobile applications, this means added complexity.
Get with the program
A quick Google search for ‘best business apps’ delivers more than a billion results. And executed properly, a mobile app will work on many levels, including the bottom line – worldwide app revenue hit more than $15bn at the end of last year. With nearly half of smart phone users downloading at least one new app every week, that’s a lot of bottom line. The key phrase here is ‘executed properly’.
As quickly as a consumer can download one app, it is just as easy to switch to another app vendor if the user experience isn’t smooth. So you need to create an app that tops the first list, avoids the second and works across all the platforms while predicting what device manufacturers will come up with next. A challenge for any organization. In this volatile environment it’s worth remembering the ground rules. Applications on mobile devices must match the functionality and performance of the operating systems and devices they’re going to be used on. At the very least, developers must tick a few boxes.
Mobile vs traditional automation
Because of the maturity of the platforms and the historical build-up of the quality teams and infrastructures, traditional test automation typically leverages more ‘black box’ testing. Mobile test automation, while conceptually the same, requires different methods to accommodate the near-constant change to platforms and architectures. Mobile testers need more access and insight into the application architecture, structure, and how it works, also known as ‘white box’ testing. To get truly flexible white box testing, a strong object recognition process must be in place.
Native object recognition is the most robust method for understanding the parameters of objects within the apps but isn’t always available to all platforms. Testers need to make sure that mobile test automation tools use a combination of object recognition technologies such as native, image based recognition, and test-based recognition. This combination will ensure that tests can be conducted on the widest variety of platforms with the least amount of maintenance across testing scripts.
While frowned on in traditional testing, instrumentation is recognized as the norm within mobile testing and is driving future thinking. So the future of mobile testing is to embrace white box testing – and to be less prescriptive about object recognition to maximize the benefits.
Make it easy to maintain
In our instant gratification society, our devices must work – and work fast.
Websites should load quickly, even on the go. The performance tester’s maxim, ‘Seven seconds can cost you $7 million dollars,’ illustrates perfectly how a delay in performance can ultimately impact the bottom line. As mobile networks spread, so does the need for robust application performance – and the risk of high profile failures increases exponentially. The rapid demand for accurate mobile application distribution across multiple devices and platforms raises the stakes for mobile development teams until performance testing is just as (if not more) critical to an applications release as functional testing. While many organizations’ current infrastructure and performance testing tools are sufficient, some fail to prepare for multi-channel/multi-speed swarms to the network under load.
Loads better?
Another, often overlooked item with mobile testing is how the application responds under heavy load. Is the user seeing the right screen? Is the application showing the correct ads under load? What about HTML5 template based applications, is the right style sheet showing up for the right device? Because there have been at least eight different versions of the Android operating system (OS) since 2009, performance and functional testing of different Android OS versions on multiple mobile devices is difficult. Add in iOS, Bada, BlackBerry.
Windows Mobile and HTML5 and testing complexity reaches another level. Testers must, therefore, use tooling that has mobile performance simulation capabilities on top of having robust object recognition technology for functional testing. Mobile testers should also prepare for the future by practicing ‘edge testing’.
Work better
Clearly, with developers under pressure to deliver more robust code that can meet all these challenges, they’re going to have to work better; find better, more productive working practices that create an improved product. Agile testing certainly ticks a lot of boxes Testers are writing automated tests to show that the software works, rather than obsessing about its vulnerability.
Of course, agile testers will still – and should – find bugs, and pre-Agile testers must write tests to show what works. But agile thinking has prompted a new way of thinking that, like the mobile technologies they support, is more fluid and interactive. Using tools that enable teams to collaborate out of the box is imperative to working better. Having a mobile testing infrastructure that allows for developers to write unit tests using open source frameworks while working with tooling that takes care of the ‘heavy lifting’ across different devices will shrink testing cycles.
The future of mobile testing isn’t – and cannot be – about silos of developers working on niche projects that seek to find problems. It’s going to be teams of developers and testers working closer together and drawing on their creativity and expertise to create robust code that meets the challenges of mobile.
Nitin Dang, country general manager, India & SAARC, Micro Focus
[email protected]
