Mobile Application Testing

Mobile testing is an ongoing process of testing the functionality and usability of mobile applications. 

Testing mobile applications provide the availability of special tools and techniques for testing. The diversity of mobile technologies, platforms, and devices creates additional challenges in developing and testing mobile applications.

Android is a free operating system designed for mobile phones, smartphones, and communicators based on Linux OS. The operating system allows you to develop Java applications that allow you to control the device. 

The apk format (file name.apk) has all the installation files of applications for the Android OS.

Functional components of Android: 

• Application framework is a set of components for various applications. 
• Dalvik virtual machine is a virtual machine in which applications run.

Main characteristics: 

• The built-in browser is powered by open-source WebKit.
• Optimized graphics with 2D library, 3D graphics
• Support for media formats: sound, video, pictures
• GSM standard, Bluetooth, 3G and WiFi, camera, GPS, compass and accelerometer.

iOS is a mobile operating system for smartphones for electronic tablets, portable players, Apple iPhone, iPod touch, and some other devices.

The operating system is characterized by the following features: 

• The speed of work, and the system interface practically do not slow down.
• The system boots up pretty quickly.
• The interface is quite colorful and clear
• The program removal system is convenient and allows you to remove programs in 2 clicks.
• You can buy any program. The catalog of programs in the AppStore is huge.
• Pretty good updates. Naturally, in each new version, there are certain errors, however, with each new version, the system becomes more convenient and functional.

Ipa file is a program file for installation on iOS. The system has a built-in Safari browser. The latest OS version is iOS 11. A new version is released once a year.

An .ipa file is a program file for installation on iOS. The system has a built-in Safari browser. The latest OS version is iOS 11. A new version is released once a year.

Disadvantages of the Apple iOS system:

• As such, there is no normal multitasking – music, radio, uploading and downloading work in the background. But not in all applications. When the application is minimized, it works for a while, and then stops.
• The operating system is closed. You cannot view the list of operating system files and use the device as a USB flash drive. This is at the same time an advantage. iOS is the most secure system in the world.
• The high cost of phones and tablets on this operating system.

Advantages:

• The largest app store with fairly high-quality applications. 
• The speed of the system (compared to others). 
• Good quality Apple phones and tablets. 
• Fast response to errors and the absence of viruses. 
• The beauty of the interface and graphics. 
• Continuous system update (once a year) incl. and for older devices.

Points to be tested

1. Screen size and touch interface:

1.1. All elements should be of such a size that the user can unambiguously click on them.

1.2. No blank screens in the application – the user should not be transported to a situation in which it is not obvious what is happening and what is being done.

1.3. You should check for multiple quick clicks on the button – often this can cause the application to crash. You should also check multitouch – pressing several buttons at the same time.

1.4. The presence or absence of “native” gestures (pinch-to-zoom, double-tap) should be checked – if, for example, zooming of a part of the application is supported, then the default gesture should be used. And if there is no need to select a picture, then it should not be highlighted by double-tap.

1. Device resources:

2.1. Memory leaks – appear on windows with a lot of information (long lists as an example), during tasks with long workflow (when the user does not exit the application for a long time), when image caching does not work correctly.

2.2. Handling situations of lack of memory for the operation of the OS, when the application is active or running in the background.

2.3. Lack of space to install or run the application.

2.4. Lack of functions supported by the application in some devices (3G, SD card).

2.5. Installing or transferring an application to an SD card.

1. Various screen resolutions and OS versions:

3.1. Retina and regular screens. On retina screens, interface elements and text are displayed smaller. Images for the retina screen can get into the non-retina version and then they will be too large.

3.2. Adaptation of the application to portrait and landscape orientations of the device.

3.3. OS versions. The application must not be installed on unsupported devices. Mandatory check on all available supported devices.

3.4. Support for required media files by this model and OS, because some developers may cut support for working with some formats.

3.5. Compliance of the views used in the application with their semantic purpose and platform concepts. Design decisions that make sense for one platform may look and feel out of place in the context of another platform.

1. Application response to an external interrupt:

4.1. Incoming and outgoing SMS, MMS, calls, notifications of other applications. 

4.2. Turning off the device, removing the battery, discharging the device.

4.3. Switching to standby mode (including with password protection). Change the orientation of the device in standby mode. 

4.4. Disconnecting and connecting the wire.

4.5. Disable and enable network, Bluetooth, airplane mode, GPS.

4.6. Loss of connection with the server or proxy (there is a connection, but the packets do not reach).

4.7. Disconnecting and connecting an SD card, and additional devices such as a physical keyboard or headset. 

4.8. Charging the device, working with a physical keyboard.

1. Paid content within the app:

5.1. Correspondence of the price and the content declared in the application. 

5.2. Purchase restore (app update)

1. Internationalization (test both in portrait and landscape mode):

6.1. Checking the correctness of the translation. 

6.2. Checking that all labels are included in the appropriate forms, buttons, etc. 

6.3. Checking date formats, separators in numbers, localization specifics (like a space before a question mark in French, superscripts “o” and “a”, in ordinal numbers in Spanish and other non-trivial moments).

1. Updates: 

7.1. Make sure that the same OS versions are supported as the previous version (if the new version of the application uses new OS features, then a stripped-down version of the application must be created for the old supported OS versions).

7.2. Checking for an adequate update (all user data is saved, etc.).

1. Constant user feedback: 

8.1. All clickable elements must have a clicked state (response to an action). In Android applications, elements can have one more state-focused.

8.2. Button response to pressing. The response speed of the elements must be sufficiently high. It is advisable to use the weakest supported devices to check this item.

8.3. Messages when loading content or progress bar. 

8.4. Messages in case of network access error, GPS.

8.5. The presence of understandable messages when trying to delete important information. 

8.6. The presence of a screen or message at the end of a process or game. 

8.7. The presence and synchronism of sounds or vibrations with notifications and other events on the screen.

1. Gestures in mobile devices:

9.1. Tap – briefly touch the surface with a fingerprint.

9.2. Double-tap – rapidly touch the surface twice with a fingerprint.

9.3. Drag – move fingerprint over the surface without losing contact.

9.4. Flick – quickly brush the surface with the fingerprint.

9.5. Pinch – touch the surface with two fingers and bring them closer together.

9.6. Spread – touch the surface with two fingers and move them apart.

9.7. Press – touch surface for an extended period of time.

9.8. Press and Tap – press the surface with one finger and briefly touch the surface with the second finger.

9.9. Press and Drag – press the surface with one finger and move the second finger over the surface without losing contact. 

9.10. Rotate – touch the surface with two fingers and move them in a clockwise or counterclockwise direction. 

On what devices and OS to test

The issue can and should be considered comprehensively and from several angles. 

First, you need to build on the product. We make a product for users, they have devices and this product should work for them. I think it’s easy to draw a simple conclusion: you need to understand what devices/platforms our audience uses when interacting with our product. How to do it:

1. Look in the technical task/requirements/any text that describes our audience or system requirements for the product. It should help a little if there is such a document. 
2. Having plunged into the modern world of flexible methodologies and not finding normal documentation, we go to the project manager, for example, and begin to communicate with him. We will find out which device pool we support, which versions (OS, browser, architecture, video processor, memory, something else where the product can be launched), and everything about our audience.
3. We look at the statistics of the use of the product – who, where and on what devices. If there is none or the product did not enter the market, then we estimate according to general statistics.

We put together the pieces of the information we found, arrange it by priorities, equivalence classes and popularity of use, form the necessary pool of devices, and go to the lead / PM or another manager for whom this is important. We get an approval and wait for the devices… or we don’t, and our whole analysis breaks down on the sharp rocks of reality. Because there is no money. Or because “why should we buy this device, it’s almost like the one, what’s the difference?”.

And here comes the second. Secondly, it is necessary to take into account the capabilities of the company. Great if you can convince to buy all the selected devices. But if the customer cannot buy all the declared devices, then what to do?

1. Most likely a couple of basic devices can still be obtained. To determine them, it is necessary to take the most popular and critical ones from our analysis.
2. Oddly enough, but we go and collect devices from colleagues. For example, go through the phones of colleagues, see what models are available, and agree that you will sometimes pick them up for testing.
3. Look towards cloud device farms, you can find temporary free options
4. Emulators. You should not rely on them completely, but it’s better than nothing.

Thirdly, the question “on what devices to test?” depends very much on the goals of testing, which in turn depend on the stages of development and testing. What we talked about above is more suitable for the product launch stage. This stage requires full and complex testing on devices.

After the launch of a product, it often moves to the support stage, patches, new versions, etc. are released. This is where our focus shifts from audience analysis, statistics, and so on. to the analysis of changes in the functionality and new features. For example, why check smartphones with different processors if there were only visual edits? It is much more important to test different screens and resolutions. Similarly, when we review bug fixes, we only take devices that could potentially be affected by the fixes.

The same can be said about the localization of defects. When finding a bug, you need to understand on what configuration it is reproduced. After all, the better the steps are localized, the easier and more accurate the correction will be. What devices to watch? Again, it depends on the implementation of the functionality.

A few tips:

1. Do not forget to update the pool of devices, at least once every six months, and statistics too. 
2. Follow the release of new versions of software and devices on which your products are launched.
3. Check out support if you haven’t already. Find out from which devices complaints are most often received. There may be sudden revelations. 
4. Don’t forget the effect of the Pesticide. If circumstances and opportunities allow, change devices sometimes.
5. Request the assistance of a professional mobile app testing company such as www.testing4success.com

Mobile Application Types 

There are three main types of mobile apps: mobile web apps, native apps, and hybrid apps.

A mobile web application is a website that opens on a gadget (smartphone or tablet) using a mobile browser.

Advantages of mobile web applications: 

• Simple development. 
• Easy access. 
• Simple update. 
• Mobile web applications do not require installation.

Disadvantages of mobile web applications: 

• No support for standalone features.
•  Limited functionality compared to hybrid and native applications (no access to the file system and local resources) 
• Redistribution Issues: Google Play and the App Store do not support the redistribution of mobile web apps.

A native app is an app designed specifically for one platform (Android, iOS, BlackBerry). 

Advantages of native applications: 

• The native app works offline.
• It can use all the functions of its device. 
• Advanced user interface. 
• Push notifications for user convenience.

Disadvantages of native apps: 

• Native app development is more expensive than mobile web apps. 
• Requires high maintenance costs.

A hybrid app is a combination of native and mobile web apps. 

It can be defined as displaying the content of a mobile site in an app format. 

Advantages of hybrid applications: 

• More cost-effective compared to the native app. 
• Simple distribution. 
• Built-in browser. 
• Device features.

Disadvantages of hybrid applications: 

• Works not as fast as the native app. 
• Graphics are less adapted to the OS in comparison with the native application.