Abstract
This chapter addresses issues related to usability and user experience of mixed reality (MR) systems based on a naturalistic iterative design approach to the development of MR applications. Design and evaluation of MR applications are still mostly based on methods used for development of more traditional desktop graphical user interfaces. MR systems are in many aspects very different from desktop computer applications, so these traditional methods are not sufficient for MR applications. There is a need for new approaches to user-centred design and development of MR systems. One such approach is based on the concepts of cognitive systems engineering (CSE). In this chapter we show how this approach can be applied to the development of MR systems. Two case studies are described, where a holistic CSE approach to design, implementation and evaluation has been used. The results show that allowing real end users (field/domain experts) to interact in a close to naturalistic setting provides insights on how to design MR applications that are difficult to attain otherwise. We also show the importance of iterative design, again involving real end users.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
All participant excerpts are translated from Swedish to English.
- 2.
References
Azuma, R.: A survey of augmented reality. Presence 6(4), 355–385 (1997)
Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., MacIntyre, B.: Recent advances in augmented reality. IEEE Computer Graphics and Applications 21(6), 34–47 (2001). http://computer.org/cga/cg2001/g6034abs.htm
Billinghurst, M., Kato, H.: Collaborative augmented reality. Communications of the ACM 45(7), 64–70 (2002)
Clark, H.H.: Using Language. Cambridge University Press, Cambridge (1996)
Davis, F.D.: Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly 13(3) (1989)
Dekker, S., Hollnagel, E.: Human factors and folk models. Cognition Technology Work 6(2), 79–86 (2004). DOIhttp://dx.doi.org/10.1007/s10111-003-0136-9
Dünser, A., Grasset, R., Billinghurst, M.: A survey of evaluation techniques used in augmented reality studies. Tech. Rep. Technical Report TR2008-02, Human Interface Technology Laboratory New Zealand (2008)
Engeström, Y., Miettinen, R., Punamaki, R. (eds.): Perspectives on activity theory (Learning in doing social, cognitive and computational perspectives). Cambridge University Press, Cambridge (1999)
Fiala, M.: Artag rev2 fiducial marker system: Vision based tracking for AR. In: Workshop of Industrial Augmented Reality, Wienna Austria (2005)
Fuhrmann, A., Löffelmann, H., Schmalstieg, D.: Collaborative augmented reality: Exploring dynamical systems. In: R. Yagel, H. Hagen (eds.) IEEE Visualization 97, pp. 459–462. IEEE (1997)
Gabbard, J.L., Swan II, J.E.: Usability engineering for augmented reality: Employing user-based studies to inform design. IEEE Transactions on Visualization and Computer Graphics 14(3), 513–525 (2008)
Glesne, C., Peshkin, A.: Becoming qualitative researchers: An introduction. Longman White Plains, New york (1992)
Granlund, R.: Web-based micro-world simulation for emergency management training. Future Generation Computer Systems 17, 561–572 (2001)
Grasset, R., Lamb, P., Billinghurst, M.: Evaluation of mixed-space collaboration. In: ISMAR ’05: Proceedings of the 4th IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 90–99. IEEE Computer Society, Washington, DC, USA (2005). DOI http://dx.doi.org/10.1109/ISMAR.2005.30
HITLAB: http://www.hitl.washington.edu/artoolkit/ (2007)
Hollnagel, E., Woods, D.D.: Cognitive systems engineering: New wine in new bottles. International Journal of Man-Machine Studies 18(6), 583–600 (1983)
Hollnagel, E., Woods, D.D.: Joint cognitive systems: Foundations of cognitive systems engineering. CRC Press, Boca Raton, FL (2005)
Hutchins, E.: Cognition in the wild. MIT Press, London, England (1995). URL http://www.netlibrary.com/summary.asp?id=1687
Klein, G., Feltovich, P.J., Bradshaw, J.M., Woods, D.D.: Common ground and coordination in joint activity, vol. Organizational Simulation, Chap. 6, pp. 139–184. John Wiley & Sons, New York (2005). http://dx.doi.org/10.1002/0471739448.ch6
Liao, Z., Landry, R.: An empirical study on organizational acceptance of new information systems in a commercial bank. In: HICSS (2000). URL http://computer.org/proceedings/hicss/0493/04932/04932021abs.htm
Lincoln, Y.S., Guba, E.G.: Naturalistic Inquiry. Sage, Beverly Hills, CA (1985)
Livingston, M.A.: Evaluating human factors in augmented reality systems. IEEE Computer Graphics and Applications 25(6), 6–9 (2005). URL http://doi.ieeecomputersociety.org/10.1109/MCG.2005.130
Milgram, P., Kishino, F.: A taxonomy of mixed reality visual displays. IEICE Transactions on Information Systems E77-D(12) (1994)
Neisser, U.: Cognition and reality. W.H. Freeman, San Francisc, CA (1976)
Nielsen, J.: Usability Engineering. Academic Press, Boston, MA (1993)
Nilsson, S., Johansson, B.: User experience and acceptance of a mixed reality system in a naturalistic setting – A case study. In: ISMAR, pp. 247–248. IEEE (2006). URL http://dx.doi.org/10.1109/ISMAR.2006.297827
Nilsson, S., Johansson, B.: Fun and usable: augmented reality instructions in a hospital setting. In: B. Thomas (ed.) Proceedings of the 2007 Australasian Computer-Human Interaction Conference, OZCHI 2007, Adelaide, Australia, November 28–30, 2007, ACM International Conference Proceeding Series, vol. 251, pp. 123–130. ACM (2007). http://doi.acm.org/10.1145/1324892.1324915
Nilsson, S., Johansson, B.: Acceptance of augmented reality instructions in a real work setting. In: M. Czerwinski, A.M. Lund, D.S. Tan (eds.) Extended Abstracts Proceedings of the 2008 Conference on Human Factors in Computing Systems, CHI 2008, Florence, Italy, April 5-10, 2008, pp. 2025–2032. ACM (2008). http://doi.acm.org/10.1145/1358628.1358633
Patton, M.Q.: Qualitative evaluation and research methods, (2nd edn). Sage Publications, Thousand Oaks, CA (1990)
Qvarfordt, P., Jönsson, A., Dahlbäck, N.: The role of spoken feedback in experiencing multimodal interfaces as human-like. In: Proceedings of ICMI‧03, Vancouver, Canada (2003)
Reichardt, C., Cook, T.: Beyond qualitative versus quantitative methods. Qualitative and Quantitative Methods in Evaluation Research pp. 7–32 (1979)
Schmalstieg, D.: Rapid prototyping of augmented reality applications with the studierstube framework. In: Workshop of Industrial Augmented Reality, Wienna, Austria (2005)
Shneiderman, B.: Designing the user interface (3rd edn). Addison Wesley (1998)
Suchman, L.A.: Plans and situated actions. Cambridge University Press, Cambridge (1987)
Sutherland, I.E.: A head-mounted three-dimensional display. In: AFIPS Conference Proceedings, vol. 33, pp. 757–764 (1968)
Swan II, J.E., Gabbard, J.L.: Survey of user-based experimentation in augmented reality. In: 1st International Conference on Virtual Reality, Las Vegas, NV (2005)
Träskbäck, M.: Toward a usable mixed reality authoring tool. In: VL/HCC, pp. 160–162. IEEE Computer Society (2004). URL http://doi.ieeecomputersociety.org/10.1109/VLHCC.2004.60
Tushman, M.L., OReilly III, C.A.: Ambidextrous organizations: Managing evolutionary and revolutionary change. California Management Review 38(4), 8–30 (1996)
Vygotsky, L.S.: Mind in society. Harvard University Press, Cambridge (1978)
Woods, D.D., Roth, E.M.: Cognitive engineering: human problem solving with tools. Human Factors 30(4), 415–430 (1988)
Acknowledgements
This research is funded by the Swedish Defence Materiel Administration (FMV). The MR system was developed in close cooperation with XM Reality AB. We are deeply indebted to all the participants in our studies who volunteered their time and expertise to these projects.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag London
About this chapter
Cite this chapter
Nilsson, S., Johansson, B., Jönsson, A. (2010). A Holistic Approach to Design and Evaluation of Mixed Reality Systems. In: Dubois, E., Gray , P., Nigay, L. (eds) The Engineering of Mixed Reality Systems. Human-Computer Interaction Series. Springer, London. https://doi.org/10.1007/978-1-84882-733-2_3
Download citation
DOI: https://doi.org/10.1007/978-1-84882-733-2_3
Published:
Publisher Name: Springer, London
Print ISBN: 978-1-84882-732-5
Online ISBN: 978-1-84882-733-2
eBook Packages: Computer ScienceComputer Science (R0)