Testers consistently ask on forums “Anyone know a tool to test Mobile Applications?” Whether on an iPhone, Android, Blackberry, iPad or any other tablet, the question consistently appears as if one expects to find a one right answer, one size fits all. By “all” what do I mean? I do not mean all types of devices or operating systems. By “all” people seem to be seeking the one answer to fit all kinds of tests for a mobile application.
Let’s step back for a minute here and think about what it is we want to test. This isn’t just the application. This is an entire system; a client/server application all in one with a data storage mechanism and its own network communication system to send data to another server. It’s a device with its own power structure, has a unique operating system with drivers affecting software and software affecting driver behavior. Is there one tool on the market now which can handle all the tests to perform this end to end testing? Can a tool “see” when an LED light goes on to indicate communication or notification? Sure a tool can, but can the tool combine other GUI usability tests, network package contents, memory usage, CPU speed and comment on network performance while changing up the variables of say the CPU speed?
Breaking down the types of tests needed for such a system is a good strategy prior to purchasing any tool. The types of testing go beyond application functionality. Most non mobile application testing includes network communication tests, data integrity tests, usability testing, performance in how the application sends and receives data, store and searches data, general application functionality. Much of this testing can be done with a tool or two, but let’s break down a mobile application scenario.
Mobile devices cannot be treated the same as any desktop application. There are added variables. Think about using your own cell phone. You’re out and about, trying to use an application, maybe a GPS type application and your cell phone rings. But you can’t answer the phone because the application won’t recognize the cell phone software to appear for you to respond. Would you say the application failed?
The application was released because it worked within its own client container… the automation tool didn’t find any problems with the application’s functionality so it should be okay to release, right? How could a GUI functional tool simulate a test as I described in the above scenario? Should the same tool be able to test this scenario? After all, the mobile application should work as a user would experience the mobile device application, right? But did anyone think about testing such a scenario? Of course there are many tests to consider beyond being able to answer a phone call while the software application is in use. Planning out these tests take time. Being able to even consider the possibilities is the piece of the puzzle often missing when mobile software applications are being tested.
In my next blog post, I will break down an actual experience of testing a mobile medical device’s software application while charging a device. Keeping a device charged is one aspect, but what about as the battery is draining, what can we expect in the software’s behavior? I will share examples of tests for inspiration of further test case generation.