Usability Evaluation of Mobile Applications

Usability Evaluation of Mobile Applications

Nidhi N Patel Pankaj Dalal

M.Tech (S.E.) Scholar Associate Professor Shrinathji Institute of Technology & Engineering, Nathdwara-313301

Abstract

During recent years the usage of mobile devices has increased greatly as new mobile technology allows the users to perform more tasks in a mobile context. This increase in usefulness makes it compulsory to evaluate the usability of mobile application and mobiles as well. Usability evaluation (UE) is an important step in software development in order to improve certain aspects of the system. However, it is often a challenge especially when it comes to evaluating applications running on mobile devices because of the restrictions posed by the device and the lack of supporting software available to collect the necessary usability data. Usability testing is a common method used to evaluate the usability of a mobile application in a development. The aim of this paper is to review some work conducted in the field of UE of Mobile Application.

Keywords

Usability Testing, Usability Metrics, Methods for evaluating usability, Attributes of Usability

1. Introduction

   The turn of this century has marked rapid growth in smart-phone market. Mobile Applications are now available for almost all areas of service as many business houses have deployed mobile applications due to competitive environment. The mobile application market has become competitive due to increase in no of providers. This makes it even more complex for developer to develop an accurate, useful & adoptable application. To ensure that the mobile application is accurate & useful one, need to evaluate the usability of mobile applications. Evaluating usability means to measure usability of mobile applications. Rachel, Derek and David has mentioned that measuring usability of mobile applications is again challenging issue because of the relatively small screen, different display resolution, limited processing power & speed,

   connectivity and limited input modalities, as all these factors have a great impact on usability of mobile applications[1].

   This paper aims to review previous studies and current techniques for usability evaluation through systematic literature review (SLR). The analysis of current techniques and previous study will result in a set of selected usability guidelines for mobile applications. In the next section, a review of several usability evaluation techniques will be presented and highlights the limitations and advantages of the various techniques. Finally, the conclusion will take place.

2. Related Study

   1. Bernhaupt has classified following Usability Evaluation (UE) methods[6] :

      • User testing (in the laboratory and the field)

      • Inspection oriented methods (like heuristic evaluation and cognitive walkthrough)

      • Self-reporting and inquiry oriented methods (like diaries and interviews)

      • Analytical modeling (task model analysis and performance models)

   Ivory has also mentioned the same methods in his thesis along with techniques used in each class, summary of which is shown in Fig -1[3].

   Bassfar has performed a comparative study to find out the different sorts of usability evaluation methods for mobile application by conducting a comparative study involving different previous researches conducted in both field and laboratory environments and found that the most commonly used methods for UE of mobile applications are: heuristic evaluation, cognitive walk-throughs evaluation, conventional user test, laboratory testing, and field testing and he has also presented a table of comparison of all these methods in terms of author, object, and assign[2].

   Figure 1 – Summary of Methods used for UE

   The usability evaluation methods for mobile application differ from one application to another based on the level of complexity [2]. Also we can conduct the usability tests in laboratories or in real scenarios which is known as field test. Kaikkonen has performed a comparative study of laboratory test & field test and found out that field test is more time consuming then laboratory test, and field test should be performed in conjunction with field pilot for some special cases where investigation of user behaviour is most important. He also mentioned following points as result of comparison of laboratory test v/s field test [5]:

   1. Both laboratory & field test gave up to 46% identical results, while there was no difference in the number of problems that occurred in the two test settings.

   2. On average the problems in the field were not more severe than the ones coming in laboratory.

   3. Field-testing is a more time consuming method than the laboratory testing.

   4. The location seemed to have a greater impact on qualitative findings of the test.

      If it is possible to create a realistic laboratory setup including elements of context then more usability problems could be found in the field compared to the lab. To cope with the

      shortcomings of testing in the laboratory, several methodological variations and combinations of various methods have been proposed.

      Zhang and Adipat have identified nine attributes that are most often evaluated: learnability, efficiency, memorability, user errors, user satisfaction, effectiveness, simplicity, comprehensibility and learning performance. All nine of them are well defined and extensively used measures of usability in more traditional desktop applications [8]. But when it comes to mobile applications as suggested by Nikolaos various aspects related with mobility have to be considered [7]. While in traditional usability studies a common assumption is that the user is performing only a single task and can therefore concentrate completely on that task. The mobile usage context users will often be performing a second action in addition to using the mobile application. For example it is quite possible that user is walking while using application on mobile. This makes user give less attention for using application as it requires the user to perform cognitive processing. The usability of application also be measured in this situation and supported by Rachel [1].Rachel has suggested a new Usability Model PACMAD Model. The PACMAD model incorporates cognitive load, which is overlooked in existing usability models and cognitive load directly impacts the usability of mobile applications [1]. The cognitive load refers to the amount of cognitive processing required by the user to use the application.

      The process of selecting appropriate usability attributes to evaluate a mobile application depends on the nature of the mobile application and the objectives of the study. Nikolaos has taken example of Mobile Guide application to explain this and suggested some measures like: Route taken and distance travelled, Percentage preferred walking speed (PPWS, User satisfaction and preferences and Experimenter observations [7]. Nikolaos has concluded a fact related with this study, is that there seem to be lack of consideration on issues of mobility and the effect of the mobility dimension on the user experience[7]. Again this is solved to an extent by the PACMAD model of Rachel [1]. The PACMAD usability model for mobile applications identifies three factors (User,

      Figure 2 – PACMAD Model [7]

      Task and Context of use shown in Fig-2). These factors are to be considered in designing mobile applications for improvement of usability.

      The word context refers hereto the user environment. The context refers to a physical location and also includes other features like the users interaction with other people or objects and other tasks.

      The model identifies seven attributes – Effectiveness, Efficiency, Satisfaction, Learnability, Memorability, Errors and Cognitive load. Each of these attributes has an impact on the overall usability of the application. These can be used to assess the usability metrics.

      The next challenge is the collection of required data for measuring the metrics. The traditional tools used in UE of desktop applications cannot be applied to small screen of mobile devices and most of the time the user is sleeping off the screen. The external camera cannot be used to capture the user activities like desktop applications. An alternative is to use screen capture software similar to the desktop. However, because of limitations posed by mobile devices, it is quite a challenge to find such applications that can accurately and efficiently capture user interaction with the mobile applications being tested [1]. A framework simplify the tasks involved in collecting usability information for mobile applications was proposed by Florence [4]. As per framework the tasks performed by the developer can be grouped into four phases [4]:

      Preparation (prepare the application prototypes to enable logging of information necessary for usability evaluation).

      Collection ( make sure that the system is able to collect the necessary data ).

      Extraction (extract all the logged data & send it to other applications for further analysis).

      Analysis (get the processed information from the extraction phase and analyse it to find out with which parts of the system the users had difficulty while interaction & how to improve it )[4].

      After data collection next step of evaluation is to select appropriate metrics. A numbers of models other then PACMAD are available for measuring usability like QUIM- Quality in Use Integrated Model developed by Ahmed [10]. QUIM is a consolidated model for usability measurement and metric. The model consists of 10 factors which are subdivided into 26 criteria. The model provides 127 metrics for the measurement of the criteria. However, the model is not optimal yet and needs to be validated. While the MUSiC – Metrics for Usability Standards in Computing developed by Bevan and MacLeod and integrated into the original ISO 9241 standard [12]. MUSiC framework has given usability metrics like effectiveness, task effectiveness, efficiency ( user efficiency & corporate efficiency ), productive & unproductive time period, etc. Software Usability Measurement Inventory (SUMI) developed by Kirakowski & Corbett is a part of MUSiC project[11]. SUMI was developed to provide measures of global satisfaction of five more specific usability areas, including effectiveness, efficiency, helpfulness, control, and learnability. Azham and Maria have suggested a new approach for developing usability metrics, where they have applied GQM(Goal Question Metric ) approach for developing usability metrics and the resulted metrics are [9]:

      • Effectiveness

      • Efficiency

      • Satisfaction

   All these metrics combinely covers usability guidelines related to simplicity, accuracy, time taken , features, safety & attractiveness. However, this model needs to be validated for future work to ensure all metric we created are applicable to mobile application.

3. Conclusion :

   Based on the UE Methodologies and its challenges for mobile application discussed above, a list of practical guidelines is proposed below, which should be carefully considered while testing a mobile application :

   • Based on type of mobile application proper methodology for UE should be selected.

   • Based on application type, user background and UE method lab testing or field testing should be performed.

   • One must consider cognitive load as usability attribute.

   • Special care must be taken for collecting user interaction data for UE.

   • Proper Metrics should be selected for measuring usability attributes.

   • Human Efficiency, User Efficiency, Relative User Efficiency, Layout Appropriateness, etc. Metrics should be given more importance in field testing.

   • Essential Efficiency and Temporal Efficiency metrics should be given more importance in laboratory testing.

4. References :

   1. Rachel Harrison, Derek Flood and David Duce, Usability of mobile applications: literature review and rationale for a new usability model, 2013

   2. Bassfar Zaid, Rozinah Jamaludin, Bajaba Wafaa, A Comparative Study of Usability Methods for Mobile Applications,2012

   3. Melody Yvette Ivory, An Empirical Foundation for Automated Web Interface Evaluation ,2001

   4. Florence Balagtas-Fernandez, Heinrich Hussmann, A Methodology and Framework to Simplify Usability Analysis of Mobile Applications , 2009

   5. Anne Kaikkonen, Aki Kekäläinen, Mihael Cankar, Titti Kallio, and Anu Kankainen, Usability Testing of Mobile Applications: A Comparison between Laboratory and Field Testing, Journal of Usability Studies-2005.

   6. R. Bernhaupt, et al., "Usability evaluation methods for mobile applications," Handbook of research on user interface design and evaluation for mobile technology, vol. 2, pp. 745-758, 2008.

   7. Nikolaos Avouris, Georgios Fiotakis and Dimitrios Raptis, On Measuring Usability of Mobile Applications in the Proceedings of the International Workshop on Meaningful

      Measures: Valid Useful User Experience Measurement (VUUM) , 2008.

   8. Zhang, D, & Adipat, B. (2005). Challenges, methodologies, and issues in the usability testing of mobile applications. International Journal of Human-Computer Interaction, 18(3), 293308,2005.

   9. Azham Hussain, Maria Kutar, Usability Metric Framework for Mobile Phone Application, 2009.

   10. Ahmed, S., Mohammad, D., Rex, B.K., and Harkirat, K.P.: Usability measurement and metrics: A consolidated model, Software Quality Control, 2006, 14, (2), pp. 159-178

   11. Kirakowski, J., and Corbett, M.: SUMI: the Software Usability Measurement Inventory, British Journal of Educational Technology, 1993, 24, (3), pp. 210-212

   12. Bevan, N., and MacLeod, M.: Usability measurement in context, Behaviour & Information Technology, 1994, 13, (1), pp. 132

-145.