Article

Usability evaluation for mobile device: a comparison of laboratory and field tests

Authors:

Henry Been-Lirn Duh,

Gerald C. B. Tan,

Vivian Hsueh-hua ChenAuthors Info & Claims

MobileHCI '06: Proceedings of the 8th conference on Human-computer interaction with mobile devices and services

Pages 181 - 186

https://doi.org/10.1145/1152215.1152254

Published: 12 September 2006 Publication History

Abstract

Usability testing of mobile devices is an emerging area of research in the field of Human-Computer Interaction. Guidelines had been established as to how usability tests should be conducted. However, there are limitations to the effectiveness of conventional usability tests, especially for mobile devices. Mobile devices typically are used in different situations, but current evaluation method cannot uncover problems in all type of situations. Hence, this study evaluates the effectiveness of conventional laboratory usability tests. It investigated the differences between usability tests on mobile phones conducted in laboratory and real life situation. Significant differences were found, including the frequency and severity of usability problems encountered, the users' behavior, and subjective responses to the device and the interaction.

References

[1]

Channelnewsasia report on Mobile Phone Usage in Singapore, 2004. Retrieved from: http://www.channelnewsasia.com/stories/singaporelocalnews/print/65398/1/.html

[2]

Condos C., James A., Every P., and Simpson T. Ten Usability Principles for the Development of effective WAP and M-commerce services. Aslib Proceedings: New Information Perspectives, 54, 6, 2002, 345--355.

[3]

Dumas J. S., and Redish J. C. A Practical Guide to Usability Testing. Intellect, Exeter. 1999.

Digital Library

[4]

Gorlenko L., and Merrick R. No Wires Attached: Usability Challenges in the Connected Mobile World. IBM Systems Journal, 42, 4, 2003, 639--651.

Digital Library

[5]

Kjeldskov J., Graham C. A Review of MobileHCI Research Methods. Proceedings of the 5th International Mobile HCI 2003 conference, 2003, 317--335.

[6]

Kjeldskov J., Stage J. New Techniques for Usability Evaluation of Mobile Systems. International Journal of Human-Computer Studies, 60, 2004, 599--620.

[7]

Kjeldskov J., Graham C., Pedell S., Vetere F., Howard S., Balbo S. and Davies J. Evaluating the Usability of a Mobile Guide: the Influence of Location, Participants and Resources. Behavior & information Technology, 24, 2005, 51--65.

[8]

Krippendorff K. Content Analysis: An Introduction to its Methodology. Thousand Oaks, CA. 2004.

[9]

LTA (Land Transport Authority) Annual Report. 2004. Retrieved from: http://www.smrt.com.sg/annualreport2004/pdf/connectivity/rail-operation.pdf

[10]

Molich R. (2000). Usable Web Design. Ingeni ren|b ger. (In Danish).

[11]

Monk A., Fellas E. and Ley E. Hearing only one side of Normal and Mobile Phone Conversation. Behavior & information Technology, 23, 2004, 301--305.

[12]

Nielsen J. Usability Engineering. Academic Press, San Diego. CA.1993.

Digital Library

[13]

NTU's Student IT Background Survey. 2003. Retrieved from: http://www.ntu.edu.sg/CITS/Home/What+is+New/student+it+background+survey+2003.htm

[14]

Nunes L. and Recarte M. A. Cognitive Demands of Hands-free Phone Conversation while Driving. From the 3rd International Conference on Traffic & Transport Psychology, Transportation Research Part F: Traffic Psychology and Behavior, 5, 2002, 133--144.

[15]

Pascoe J., Ryan N., Morse D. Using While Moving: Human-Computer Interaction Issues in Fieldwork Environments. Annual Conference Meeting Transactions on Computer-Human Interaction, 7, 3, 2000, 417--437.

Digital Library

[16]

Ramsay M. Mildly Irritating: a WAP usability study. Aslib Proceedings: New Information Perspectives, 53, 2001, 141--158.

[17]

Rubin J. Handbook of Usability Testing. Wiley, New York. 1994.

Digital Library

Cited By

Hrast TAhlström DHitz M(2024)Observations and Considerations for Implementing Vibration Signals as an Input Technique for Mobile DevicesMultimodal Technologies and Interaction10.3390/mti80900768:9(76)Online publication date: 2-Sep-2024
https://doi.org/10.3390/mti8090076
Marcilly RMonkman HPelayo SLesselroth B(2024)Usability Evaluation Ecological Validity: Is More Always Better?Healthcare10.3390/healthcare1214141712:14(1417)Online publication date: 16-Jul-2024
https://doi.org/10.3390/healthcare12141417
Weichbroth P(2024)Usability Testing of Mobile Applications: A Methodological FrameworkApplied Sciences10.3390/app1405179214:5(1792)Online publication date: 22-Feb-2024
https://doi.org/10.3390/app14051792
Show More Cited By

Index Terms

Usability evaluation for mobile device: a comparison of laboratory and field tests
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods

Recommendations

The research framework of eye-tracking based mobile device usability evaluation
PETMEI '11: Proceedings of the 1st international workshop on pervasive eye tracking & mobile eye-based interaction

Eye-tracking is a valuable tool for mobile device usability research, but there are still many challenges about how to create good usability evaluation, such as the accurate enough eye-movement data from the small view angle on a real mobile device. ...
Usability in mobile interface browsing

The advent of mobile devices imposes great challenges for user-friendly displays for effective browsing for Web contents. This paper reports some initial findings in an empirical study that explores the problem of finding a highly-efficient, user-...
Usability Guidelines for Mobile Devices: Length of Hyperlinks
14th IFIP TC 13 International Conference on Human-Computer Interaction --- INTERACT 2013 - Volume 8120

The use of mobile devices is increasing in recent years, and their characteristics are different from those of computers keyboard and screen size, interaction method, etc., so it is important to specifically study the usability of these devices. This ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

MobileHCI '06: Proceedings of the 8th conference on Human-computer interaction with mobile devices and services

September 2006

320 pages

ISBN:1595933905

DOI:10.1145/1152215

General Chairs:
Marko Nieminen
Helsinki University of Technology, Finland
,
Mika Röykkee
Nokia Research Center, Finland

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 September 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

MobileHCI06

MobileHCI06: Human-Computer Interaction with Mobile Devices and Services

September 12 - 15, 2006

Helsinki, Finland

Acceptance Rates

Overall Acceptance Rate 202 of 906 submissions, 22%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

133
Total Citations
View Citations
7,637
Total Downloads

Downloads (Last 12 months)92
Downloads (Last 6 weeks)12

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hrast TAhlström DHitz M(2024)Observations and Considerations for Implementing Vibration Signals as an Input Technique for Mobile DevicesMultimodal Technologies and Interaction10.3390/mti80900768:9(76)Online publication date: 2-Sep-2024
https://doi.org/10.3390/mti8090076
Marcilly RMonkman HPelayo SLesselroth B(2024)Usability Evaluation Ecological Validity: Is More Always Better?Healthcare10.3390/healthcare1214141712:14(1417)Online publication date: 16-Jul-2024
https://doi.org/10.3390/healthcare12141417
Weichbroth P(2024)Usability Testing of Mobile Applications: A Methodological FrameworkApplied Sciences10.3390/app1405179214:5(1792)Online publication date: 22-Feb-2024
https://doi.org/10.3390/app14051792
Collins-Pisano CLeggett AGambee DFortuna K(2024)Usability, Acceptability, and Preliminary Effectiveness of a Peer-Delivered and Technology-Supported Mental Health Intervention for Family Caregivers of People With Dementia: Field Usability StudyJMIR Human Factors10.2196/4120211(e41202)Online publication date: 27-May-2024
https://doi.org/10.2196/41202
Mayas CSteinert TReif BKrömker HHirth M(2024)Pimp My Lab? The Impact of Context-Specific Usability Laboratory Environments in Public TransportHCI in Mobility, Transport, and Automotive Systems10.1007/978-3-031-60480-5_11(183-194)Online publication date: 1-Jun-2024
https://doi.org/10.1007/978-3-031-60480-5_11
Bringula RJose JLardizabal ALizaso J(2023)Predictors of Usability of a Mobile Intelligent Agent Information Provider for College StudentsInternational Journal of Mobile Human Computer Interaction10.4018/IJMHCI.32245715:1(1-19)Online publication date: 9-May-2023
https://dl.acm.org/doi/10.4018/IJMHCI.322457
Dongas RGrace KGillespie SHoggenmueller MTomitsch MWorrall S(2023)Virtual Urban Field Studies: Evaluating Urban Interaction Design Using Context-Based Interface PrototypesMultimodal Technologies and Interaction10.3390/mti70800827:8(82)Online publication date: 18-Aug-2023
https://doi.org/10.3390/mti7080082
Kays RGoagoses NWinschiers-Theophilus H(2023)Designing for Educational Resilience to Reduce School Dropout: A Case Study of Namibian San LearnersACM Journal on Computing and Sustainable Societies10.1145/36163891:2(1-21)Online publication date: 6-Dec-2023
https://dl.acm.org/doi/10.1145/3616389
Desmal AHamid SOthman MZolait A(2022)Exploration of the usability quality attributes of mobile government services: a literature reviewPeerJ Computer Science10.7717/peerj-cs.10268(e1026)Online publication date: 28-Jul-2022
https://doi.org/10.7717/peerj-cs.1026
Munson SFriedman EOsterhage KAllred RPullmann MAreán PLyon A(2022)Usability Issues in Evidence-Based Psychosocial Interventions and Implementation Strategies: Cross-project AnalysisJournal of Medical Internet Research10.2196/3758524:6(e37585)Online publication date: 14-Jun-2022
https://doi.org/10.2196/37585
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents