- Open Access
- Total Downloads : 3
- Authors : Niranjanamurthy M, Sai Prasad C, Manushree Sp, Asharani A, Dr. Dharmendra Chahar
- Paper ID : IJERTCONV2IS02001
- Volume & Issue : NCSE – 2014 (Volume 2 – Issue 02)
- Published (First Online): 30-07-2018
- ISSN (Online) : 2278-0181
- Publisher Name : IJERT
- License: This work is licensed under a Creative Commons Attribution 4.0 International License
Research Challenges in Mobile Application Testing
Niranjanamurthy M#1, Sai Prasad C #2, Manushree SP#3,Asharani A#4 Dr. Dharmendra Chahar #5
1 Research Scholar, Dept. of Computer Science Engg., JJTU, Rajasthan,, firstname.lastname@example.org
2Student of MCA, MSRIT, Bangalore, email@example.com 3Student of MCA, MSRIT, Bangalore, firstname.lastname@example.org 4Student of MCA, MSRIT, Bangalore, email@example.com
5 HOD. Dept. of CS & IT, Seth G. B. Podar College, Nawalgarh (Jhunjhunu), Rajasthan
Mobile application testing is mainly deals with functionality, Usability and consistency of that application. Mobile devices are everywhere. Businesses depend on them. Customers interact with them like never before. Their exponential growth is creating high demand and an even higher need for advanced functionality. But innovation and increased speed to market bring many testing challenges.Mobile applications either come pre- installed or can be installed from mobile software distribution platforms. Mobile devices have witnessed a phenomenal growth in the past few years. Device and platform diversity, short release cycles, lack of mature testing tools and the variety of network connectivity options result in frequent cost overruns and missed deadlines in todays mobile application testing environment. A comprehensive mobile testing strategy that includes device and network infrastructure, optimized selection of target devices, and an effective combination of manual and automated testing tools to cover both functional and non-functional testing is essential for getting your mobile applications to market on time and within budget.
Keywords Mobile application testing, Types of Mobile Application Testing, Challenges in Mobile App Testing, Customers Challenges in Mobile Testing, , Interruptions of the mobile application Testing, Mobility Testing Process, Security testing in Mobile application,
Mobile application testing (MAT) is a process by which application software developed for hand held mobile devices is tested for its functionality, usability and uniformity.Mobile Application Testing techniques are: Functional Testing, Laboratory Testing, Performance Testing, Memory Leakage Testing, Interrupt Testing, Usability testing, Installation testing, Certification Testing, Challenges in Mobile Application Testing are: Types of Mobile
Devices, Diversity in Mobile Platforms/OS, Mobile network operators, Scripting.
Mobile devices have witnessed a phenomenal growth in the recent years. A study conducted by the Yankee Group predicts generation of $4.2 billion in revenue by 2013 through 7 billion U.S. Smartphones app downloads. Mobile application testing involves evaluating functional & non-functional characteristics of application softwares developed for Hand held devices such as mobile phones, tablets etc. Testing a mobile application across multiple devices running on the same platform and every platform poses a unique challenge for testers.
Testing mobile applications through:
I phone — Simulator
Android — Emulator
Aim of the Study
Aims of this paper are:
To understand what is Mobile Application Testing
To Know the Types of Test techniques required for Mobile Application
Knowledge on Mobility Testing Process
To understand Key Challenges in Mobile Application Testing
To know Mobile Testing Tools and Tool Selection Criteria
To understand Customers Challenges in Mobile ApplicationTesting
To know how mobile testing isdifferent?
Knowledge on Interruptions of the mobile Application Testing
Understand the Security testing conditions in Mobile Application
The usability of mobile applications is criticalfor their adoption because of the relatively small screenand awkward (sometimes virtual) keyboard, despite therecent advances of Smartphones. Traditional laboratorybased usability testing is often tedious, expensive, and does not reflect real use cases. In this paper, we propose a toolkit that embeds into mobile applications the ability to automatically
collect user interface (UI) events as the user interacts with the applications. The events are fine-grained and usefulfor quantified usability analysis. We have implementedthe toolkit on Android devices and we evaluated the toolkit with a real deployed Android application by comparing event analysis (state- machine based) with traditional laboratory testing (expert based). The results show that our toolkit is effective at capturing detailed UI events for accurate usability analysis.
Challenges in Mobile App Testing: Its been clear for a while that mobile devices are the currentmarket players, even more so that some experts have beencounting on them to take over the PCs and Desktops in nearfuture. But as with any emerging technology, developing andimplementing mobile applications can pose a number of
Unique challenges. Mobile applications, although havelimited computing resources, are often built to be as agileand reliable as PC-based applications. In order to meet thechallenge, mobile application testing has evolved as a
Separate stream of testing.The major challenge in Mobile App Testing is themultiplicity of mobile devices with different capabilities,features and restrictions. Devices may have differenttechnical capabilities such as amount of available memory,screen resolution, screen orientation and size of the display,network connectivity options, support for different standardsand interfaces 
Figure 1.Mobile Testing
There are multiple operating systems that are prevalent in the mobile space like Symbian, Android, iPhone OS, Windows, Linux, Blackberry OS, palm OS, Brew, etc. Each of the operating systems can have further versions for different types of devices which make platform testing complex and further challenging. Another challenge is that the developers need to focus on developing applications that are easy to use on a mobile and consume less power.
Types of Mobile Application Testing
Functional Testing- Functional testing ensures that the application is working as per the requirements. Most of the test conducted for this is driven by the user interface and call flows.
Laboratory Testing- Laboratory testing, usually carried out by network carriers, is done by simulating the complete wireless network. This test is performed to find out any glitches when a mobile application uses voice and/or data connection to perform some functions.
Performance Testing- This testing process is undertaken to check the performance and behaviour of the application under certain conditions such as low battery, bad network coverage, low available memory, simultaneous access to applications server by several users and other conditions. Performance of an application can be affected from two sides: applications server side and clients side. Performance testing is carried out to check both.
Memory Leakage Testing- Memory leakage happens when a computer program or application is unable to manage the memory it is allocated resulting in poor performance of the application and the overall slowdown of the system. As mobile devices have significant constraints of available memory, memory leakage testing is crucial for the proper functioning of an application.
Interrupt Testin- An application while functioning may face several interruptions like incoming calls or network coverage outage and recovery.
The different types of interruptions are:
Incoming and Outgoing SMS and MMS
Incoming and Outgoing calls
Cable Insertion and Removal for data transfer
Network outage and recovery
Media Player on/off
Device Power cycle
An application should be able to handle these interruptions by going into a suspended state and resuming afterwards.
Usability testing- Usability testing is carried out to verify if the application is achieving its goals and getting a favourable response from users. This is important as the usability of an application is its key to commercial success (it is nothing but user friendly).
Installation testing- Certain mobile applications come pre-installed on the device whereas others have to be installed from the store. Installation testing verifies that the installation process goes smoothly without the user having to face any difficulty. This testing process covers installation, updating and uninstalling of an application.
Certification Testing- To get a certificate of compliance, each mobile device needs to be tested
against the guidelines set by different mobile platforms.
The Certified Mobile Application Tester popularly known as CMAT certification exam is offered by the Global Association for Quality Management (GAQM) via Pearson Vue Testing Centreworldwide to benefit the Mobile Application Testing Community. 
Figure 3. Testing Aspects
Mobility Testing Process
A typical end-to-end mobile testing process should start from creating test cases of the application, performing user acceptance and finally device testing stage.The stages in mobile application testing process are as follows: 
Figure 2. Testing Process
Security and data privacy are of highest importance in todays scenario. Users are worried about their privacy.
Key Challenges in Mobile Application Testing
Variety of Mobile Devices- Mobile devices differ in screen sizes, input methods (QWERTY, touch, normal) with different hardware capabilities.
Diversity in Mobile Platforms/OS- There are different Mobile Operating Systems in the market. The major ones are Android, IOS, BREW, BREWMP, Symbian, Windows Phone, and BlackBerry (RIM). Each operating system has its own limitations. Testing a single application across multiple devices running on the same platform and every platform poses a unique challenge for testers.
Mobile network operators- There are over 400 mobile network operators in the world;  out of which some are CDMA, some GSM, whereas others use less common network standards like FOMA, and TD-SCDMA. Each network operator uses a different kind network infrastructure and this limits the flow of information.
Scripting- The variety of devices makes executing the test script (Scripting) a key challenge. As devices differ in keystrokes, input methods, menu structure and display properties single script does not function on every device. 
Figure 4. Test Cycles of Desktop and Mobile applications
Some Popular Mobile Testing Tools and Tool Selection Criteria
Figure 5. Mobile Testing Tools
Tool Selection Criteria:
It is based on:
Cost Benefit Analysis
Timelinesfor the project
Regression Test Count & Complexity Scope based decisions:
Supported Versions of platforms
Types of automation supported.
Device variants within platforms (form factors) Feature based decisions:
Working on both simulator /device
Integration to test management
Ease of use and scripting supportInfrastructure Infrastructure decisions:
Dependency on Mac/desktops
Cloud based/stand alone
Figure 6. Mobile Platforms
Figure 7. Selecting Mobile Testing Tools
Customers Challenges in Mobile Testing
Variety of mobile devices in market and multiple manufacturers
Coping with the short lifecycle of the mobile application in market
Huge variety of hardware capabilities
Shorter duration of device life in the market
Variety of network modes like 2G/3G/4G/Wi- Fi/Wi-Max
Huge investment and high time frame to set up an test lab
Most of the tools are image based comparison tool
One test tool may not support all platforms versions
Tools may need jailbreak/rooting that invokes security threat
Testing on latest technologies like HTML5, NFC, etc.
High rise of localized application capability
Choice between simulators and real devices for testing
Mobile testing how is it different?
Mobile devices have different application runtimes.
Multiple type of Networks(GSM / GPRS / Wi-Fi / Wi-Max etc )
Unpredictable time taken for data transfer
Different speed of connectivity across geographies
Multiple Network Operators with customized Network features
Limitations in processing speed
Limitations of Memory size of mobile
Differences in Communication Protocols of devices WAP/ HTTP.
Interruptions of the mobile application Testing
There are various events that can interrupt the flow of your application. Your application should be able to handle these and should be tested for the same.
Other app notifications
Home screen jump
Security testing in Mobile application Security and data privacy are of highest importance in todays scenario. Users are worried about their data and recommendation being exposed through vulnerable applications.
Is your application storing payment information or credit card details?
Does your application use secure network protocols?
Can they be switched to insecure ones?
Does the application ask for more permissions than it needs?
Does your application use certificates?
Does your application use a Device ID as an identifier?
Does your application require a user to be authenticated before they are allowed to access their data?
*Is there a maximum number of login attempts before they are locked out?
Figure 8. Mobile Application Testing
Mobile application testing is a process by which application software developed for hand held mobile devices is tested for its functionality, usability and uniformity.Challenges in Mobile App Testing: Its been clear for a while that mobile devices are the currentmarket players, even more so that some experts have beencounting on them to take over the PCs and Desktops in nearfuture.For testing Mobile Application Testing we need to follow the following test techniques: Functional Testing, Laboratory Testing, Performance Testing,Memory Leakage Testing,Interrupt Testing, Usability testing, Installation testing, Certification Testing.
Tool Selection Criteria It is based on cost Benefit Analysis, Timelines for the projectRegression Test Count & Complexity and Customers Challenges in Mobile Testing, Mobile Testing How is it different? And different Interruptions and security of the mobile application Testing and platforms for mobile application testing topics are discussed.
We thank Dr. T. V. Suresh Kumar, Prof. and Head, Dept. of MCA, MSRIT, Bangalore-54. For his continuous support andencouragement for completing this research paper and alsothanks to MSRIT management.
We thank Mr. S. Jagannatha , Associate Professor. Dept. of MCA, MSRIT, Bangalore-54 for his constant support and motivation.
(Jhunjhunu), Rajasthanfor his constant support and guidance.
We thank Manish Kumar, Assistant Professor Dept. of MCA, MSRIT, Bangalore-54 for his constant support and suggestions.
Xiaoxiao Ma, Bo Yan Guanling Chen- "Design and Implementation of a Toolkit for Usability Testing of Mobile Apps"Published online: 21 November 2012 – Springer Science -Business Media New York 2012.
R RamchandraNimbalkar – "Mobile Application Testing and Challenges" – International Journal of Science and Research (IJSR), – Volume 2 Issue 7, July 2013 -India Online ISSN: 2319-7064
http:// www.tcs.com/resources/white_papers/Pages/Mobile_applic ation_testing.aspx http://en.wikipedia.org/wiki/Mobile_application_testing 
-application-testing.pdf http://www.rapidvaluesolutions.com/mobile-application- testing-step-by-step-approach/
Julian Harty, MahadevSatyanarayananA – Practical Guide to Testing Wireless Smartphones Applications (Synthesis Lectures on Mobile and Pervasive Computing)- 2009 – ISBN-10: 1608452530
Rex Black: Advanced Software Testing Vol. 1,
Shroff Publishers, 2011
Srinivasan DesikanGopalaswamy: Software Testing Principles and Practices, 5
th Edition , Pearson Education, 2007