What? Hardware Tests for the Software Tester!


Hardware Tests

Ever do any testing of software behavior while the mobile device is charging?  Ever thought to do so?  Why would changing a hardware condition affect the behavior of the software?

Welcome to the world of Mobile Device and true Mobile System testing!

If you thought testing the software behavior while charging the device was not part of a software tester’s job, well, surprise, it most certainly is!  Anything which could affect software behavior is a test to be considered.  So let’s think about what kinds of results you are looking for:  network communication uninterrupted, optimal internal database searches, memory performance, among other tests.

To understand the connection of charging the device and software behavior, the software tester should be educated in more than the functionality of the GUI belonging to the application under test.  What I mean, is understand what kinds of connections your software application make to your operating system and to other software applications.  Does your software application utilize a clock or record time?  If so, then your application communicates to the operating system.  Basically, think two applications are now under test.  Now, your application’s functions and your applications interactions with another application are two different software test types and two different thought processes for test case generation.  If your application stores data, manipulates data, repackages data and then sends data out to another device or server, you’re dealing with more than one application so your testing must be more system integration testing rather than singular application testing.

Your thought process must expand on the idea your device is an entire system.  One example of tests I discovered was charging the device and witnessing effects of software application behavior.  Charging your device and testing your software’s behavior at various points of the battery level is extremely important and often overlooked.  I discovered these tests while conducting functional tests but noticed how a low battery changed the way my software application behaved.  What evolved were a series of tests based on various battery levels and conditions of software application engagement.

One particular test came into being was when I noticed how hot my device felt to the touch after charging the device from a completely drained battery.  I also noticed with the heat, my software started to behave erratic.   The data my device had analyzed and repackaged for network travel appeared unrecognizable once the data reached its destination.   Sometimes data would not get to its end destination due to the device itself powering down completely and not recoverable.

My battery level variables were:  0%, 10%, 20%, 50%, 60%, 75% and 90%.  Conditions had to also be tested with each of these levels of battery charge.  These conditions included:  software application installed but not running, software application installed and running while not engaging directly with the GUI and CPU speed was low, software application installed and running with GUI engaged and CPU speed at its highest rate, and, no software application installed but operating system was installed and Windows Explorer was engaged.

I began tests by checking the clock, LED lights making sure all worked as expected and would use the application to create communication with and without the GUI being engaged.  What I found was when the application was engaged, the CPU speed increased and created even more heat.  When the battery generates heat up to 70C degrees, even 65C degrees, the cell modem can be damaged as well as other components contained within the device itself.  It’s a small contained space, and there is very little opportunity for the heat to dissipate.

With these tests, we see the direct effect of hardware has on software and in turn software affects hardware behavior.  This is one reason why it’s critical for a software tester to understand how hardware engages with its operating system which affects software behavior and vice versa.  Software testers must now create test cases beyond the software application itself.

More investigation had to be done with regards to resolving the problem.  In our software we put in a check, to report the battery temperature every 5 minutes as well as CPU speed.   Between my operating system’s log and the application’s log, I was able to test what was happening when.  I could use variable temperatures to conduct tests and see what happened with the network communication.  I could also how fast the CPU was spinning when storing data or retrieving data all the while keeping watch of the temperature.

How we were able to control the temperature was to take the recorded temperature and apply an If / Then conditional where If the recorded temp > than 60C degrees, then the  cell modem would shut down and not generate further heat by trying to transmit data.  Of course the software application needed to find a way to turn the cell modem back on, so an additional If / Then condition was added.  If the cell modem = off AND the recorded temp is < 55C degrees, then turn the cell modem back on for communication to continue.  These conditions evolved into actual requirements which of course became part of my regression testing.

Hopefully sharing some of my tests and real examples will inspire you, the mobile software application tester, to remember to conduct mobile device testing as part of your software application tests.  Include time for these tests in your test plan even if you have to “hide” it as system integration testing.  I say “hide” because management may not understand why you are testing hardware like charging a device.  But remind them when variables and conditions affect your software’s functional behavior; they become a part of software testing.

Next up:  what kinds of test cases do you create based on charging a device as a general rule?  I will talk about examples of the speed of charging a device’s battery and its effect on software.  I will also include some regression tests you can include with your own testing ideas.