Article

Évaluation des dispositifs mobiles: sur le terrain ou en laboratoire ?

Authors:

Francis Jambon,

Caroline Golanski,

Pierre-Jacques PommierAuthors Info & Claims

IHM '06: Proceedings of the 18th Conference on l'Interaction Homme-Machine

Pages 35 - 42

https://doi.org/10.1145/1132736.1132741

Published: 18 April 2006 Publication History

Abstract

This article focuses on usability testing of mobile devices "in the wild". We are interested in ensuring the methodological validity of these analyses. We so implemented an original approach which consists in carrying out a meta-evaluation of two evaluations of the same quasi-realistic experiment. The first one uses a traditional methodology similar to usability laboratory settings. The second one mimics the conditions of experiments carried out "in the wild". Our objective is to validate -in advance-the methodology for usability evaluations "in the wild".

References

[1]

1. Arhippainen, L., Rantakokko, T. et Tähti, M. Mobile Feedback Application for Emotion and User Experience Collection. Proactive computing workshop-PROW 2004 (25-26 November, Helsinki, Finland), 2004, pp. 77-81.

[2]

2. Bartek, V. et Cheatham, D. Experience remote usability testing, Part 1 & 2. IBM's Pervasive Computing Division, 2003, URL: "http://www- 106.ibm.com/developerworks/web/library/wa- rmusts1/".

[3]

3. Brush, A.J.B., Ames, M. et Davis, J. A comparison of synchronous remote and local usability studies for an expert interface. CHI'04 conference on Human factors in computing systems extended abstracts (24-29 April, Vienna, Austria), ACM Press, 2004, pp. 1179-1182.

Digital Library

[4]

4. Demumieux, R. et Losquin, P. Collecter les usages réels des clients de téléphonie mobile (un outil embarqué). 17e Conférence Francophone sur l'Interaction Homme-Machine - IHM'05 (27-30 septembre, Toulouse, France), ACM Press, 2005, pp. 293-298.

Digital Library

[5]

5. Forest, F., Guilloux, V., Mallein, P. et Panisset, J. La conception assistée par l'usage dans les bibliothèques publiques. Kœnig, M.-H. (Ed.). In Connaître ses publics : savoir pour agir (La Boîte à outils), Institut de Formation des Bibliothécaires - IFB, Villeurbanne, 1998.

[6]

6. Goodman, J., Brewster, S. et Gray, P. Using Field Experiments to Evaluate Mobile Guides. 3rd Annual Workshop on HCI in Mobile Guides (Glasgow, UK), 2004.

[7]

7. Hagen, P., Robertson, T., Kan, M. et Sadler, K. Emerging research methods for understanding mobile technology use. OzCHI'2005 (21-25 November, Canberra, Australia), ACM Press, 2005.

Digital Library

[8]

8. Hammontree, M., Weiler, P. et Nayak, N. Remote Usability Testing. interactions. 1, 3 (1994), pp. 21-25.

Digital Library

[9]

9. Hartson, H.R., Castillo, J.C., Kelso, J. et Neale, W.C. Remote evaluation: the network as an extension of the usability laboratory. SIGCHI conference on Human factors in computing systems (CHI'96) (Vancouver, British Columbia, Canada), ACM Press, 1996, pp. 228-235.

Digital Library

[10]

10. Hilbert, D.M. et Redmiles, D.F. Extracting usability information from user interface events. ACM Computing Surveys - CSUR. 32, 4 (2000), pp. 384-421.

Digital Library

[11]

11. Isomursu, M., Kuutti, K. et Väinämö S. Experience clip: method for user participation and evaluation of mobile concepts. 8th conference on Participatory design: Artful integration: interweaving media, materials and practices (Toronto, Ontario, Canada), ACM Press, 2004, pp. 83-92.

Digital Library

[12]

12. Kellar, M., Reilly, D., Hawkey, K., Rodgers, M., MacKay, B., Dearman, D., Ha, V., MacInnes, W.J., Nunes, M., Parker, K., Whalen, T. et Inkpen, K.M. It's a jungle out there: practical considerations for evaluation in the city. CHI'05 extended abstracts on Human factors in computing systems (Portland, OR, USA), ACM Press, 2005, pp. 1533-1536.

Digital Library

[13]

13. Kjeldskov, J., Graham, C., Pedell, S., Vetere, F., Howard, S., Balbo, S. et Davies, J. Evaluating the Usability of a Mobile Guide: The Influence of Location, Participants and Resources. Behaviour & Information Technology. 24, 1 (2005), pp. 51-65.

[14]

14. Kjeldskov, J., Skov, M.B., Als, B.S. et Høegh, R.T. Is It Worth the Hassle? Exploring the Added Value of Evaluating the Usability of Context-Aware Mobile Systems in the Field. Mobile Human-Computer Interaction-MobileHCI'2004 (13-16 September, Glasgow, UK), Lecture Notes in Computer Science, Springer-Verlag, 2004, pp. 61-73.

[15]

15. Kjeldskov, J. et Stage, J. New Techniques for Usability Evaluation of Mobile Systems. International Journal of Human-Computer Studies - IJHCS. 60, 5-6 (2004), pp. 599-620.

[16]

16. Roto, V., Oulasvirta, A., Haikarainen, T., Lehmuskallio, H. et Nyyssönen, T. Examining mobile phone use in the wild with quasi-experimentation. Helsinky Institute for Information Technology (HIIT), August 2004, Technical Report 2004-1.

[17]

17. Salembier, P., Kahn, J., Calvet, G., Zouinar, M. & Relieu, M. "Just follow me" - Examining the use of a multimodal mobile device in natural settings. International Conference on Human Computer Interaction - HCII'05 (22-27 July, Las Vegas, USA), Lawrence Erlbaum Associates, 2005.

[18]

18. Scholtz, J. Adaptation of Traditional Usability Testing Methods for Remote Testing. 34th Annual Hawaii International Conference on System Sciences - HICSS'01 (3-6 January, Maui, Hawaii), IEEE Computer Society, 2001, pp. 5030.

Digital Library

[19]

19. Thompson, K.E., Rozanski, E.P. et Haake, A.R. Here, there, anywhere: remote usability testing that works. 5th conference on Information Technology Education (Salt Lake City, UT, USA), ACM Press, 2004, pp. 132-137.

Digital Library

[20]

20. Tullis, T., Fleischman, S., McNulty, M., Cianchette, C. et Bergel, M. An Empirical Comparison of Lab and Remote Usability Testing of Web Sites. Usability Professionals Association Conference - UPA'2002 (8-12 July, Orlando, FL, USA), 2002.

[21]

21. Waterson, S., Landay, J.A. et Matthews, T. In the lab and out in the wild: remote web usability testing for mobile devices. CHI'02 conference on Human Factors in Computing Systems extended abstracts (Minneapolis, Minnesota, USA), ACM Press, 2002, pp. 796-797.

Digital Library

Cited By

Jambon F(2011)Experimental Setups for User Evaluation of Mobile Devices and Ubiquitous SystemsHuman-Computer Interaction and Innovation in Handheld, Mobile and Wearable Technologies10.4018/978-1-60960-499-8.ch009(155-171)Online publication date: 2011
https://doi.org/10.4018/978-1-60960-499-8.ch009

Index Terms

Évaluation des dispositifs mobiles: sur le terrain ou en laboratoire ?

Recommendations

Sami, services d'assistance mobiles et intelligents
Évaluation d'un Système pour le Résumé Automatique de Documents Électroniques
AI '01: Proceedings of the 14th Biennial Conference of the Canadian Society on Computational Studies of Intelligence: Advances in Artificial Intelligence

Cet article prÉsente une Évaluation de la fonction de rÉsumÉ du système Extractor. Quatre attributs de qualitÉ ont ÉtÉ considÉrÉs, soit la cohÉrence et la balance des rÉsumÉs produits ainsi que la polyvalence et la performance du système. Notre dÉmarche ...
Flex RIA development and usability evaluation
WISE'07: Proceedings of the 2007 international conference on Web information systems engineering

A key factor for successful Rich Internet Applications (RIA) is to provide an interface which is effective and efficient to use. This paper proposes a framework for model driven development of Adobe® Flex™ RIAs which uses task models and international ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

IHM '06: Proceedings of the 18th Conference on l'Interaction Homme-Machine

April 2006

300 pages

ISBN:1595933506

DOI:10.1145/1132736

Conference Chairs:
Jean-Marc Robert
École Polytechnique de Montréal, Québec
,
Bertland David
École Centrale de Lyon, France

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 April 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Acceptance Rates

Overall Acceptance Rate 103 of 199 submissions, 52%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
264
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Jambon F(2011)Experimental Setups for User Evaluation of Mobile Devices and Ubiquitous SystemsHuman-Computer Interaction and Innovation in Handheld, Mobile and Wearable Technologies10.4018/978-1-60960-499-8.ch009(155-171)Online publication date: 2011
https://doi.org/10.4018/978-1-60960-499-8.ch009

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten