Abstract
This chapter presents an overview of the Mixed Reality (MR) paradigm, which proposes to overlay our real-world environment with digital, computer-generated objects. It presents example applications and outlines limitations and solutions for their technical implementation. In MR systems, users perceive both the physical environment around them and digital elements presented through, for example, the use of semitransparent displays. By its very nature, MR is a highly interdisciplinary field engaging signal processing, computer vision, computer graphics, user interfaces, human factors, wearable computing, mobile computing, information visualization, and the design of displays and sensors. This chapter presents potential MR applications, technical challenges in realizing MR systems, as well as issues related to usability and collaboration in MR. It separately presents a section offering a selection of MR projects which have either been partly or fully undertaken at Swiss universities and rounds off with a section on current challenges and trends.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Weiser, M.: Some computer science issues in ubiquitous computing. Commun. ACM 36(7), 75–84 (1993)
Sutherland, I.: A head-mounted three dimensional display. In: Proceedings of the Fall Joint Computer Conference, vol. 33, pp. 757–764 (1968)
Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., MacIntyre, B.: Recent advances in augmented reality. IEEE Comput. Graph. Appl. 21(6), 34–47 (2001)
Milgram, P., Kishino, F.: A Taxonomy of Mixed Reality Visual Displays. IEICE Transactions on Information Systems 77, 1321–1329 (1994)
Feiner, S., Macintyre, B., Seligmann, D.: Knowledge-based augmented reality. In: Communications of the ACM, vol. 36, pp. 53–62. ACM Press, New York (1993)
Lee, J., Rhee, G.: Context-aware 3D visualization and collaboration services for ubiquitous cars using augmented reality. The International Journal of Advanced Manufacturing Technology 37, 431–442 (2008)
Reiners, D., Stricker, D., Klinker, G., Muller, S.: Augmented Reality for Construction Tasks: Doorlock Assembly. In: 1st International Workshop on Augmented Reality (IWAR 1998), San Francisco (1998)
Doil, F., Schreiber, W., Alt, T., Patron, C.: Augmented reality for manufacturing planning. In: Proceedings of the workshop on Virtual environments 2003, pp. 71–76. ACM Press, New York (2003)
Billinghurst, M., Kato, H., Poupyrev, I.: The MagicBook: a transitional AR interface. In: Computers & Graphics, vol. 25, pp. 745–753. Elsevier, Amsterdam (2001)
Klinker, G., Dutoit, A., Bauer, M., Bayer, J., Novak, V., Matzke, D.: “Fata Morgana” A Presentation System for Product Design. In: Proceedings of the International Symposium on Mixed and Augmented Reality (ISMAR 2002). IEEE Computer Society, Washington (2002)
State, A., Livingston, M.A., Garrett, W.F., Hirota, G., Whitton, M.C., Pisano, E.D., Fuchs, H.: Technologies for augmented reality systems: Realizing ultrasound-guided needle biopsies. In: SIGGRAPH, pp. 439–446 (1996)
State, A., Hirota, G., Chen, D.T., Garrett, W.F., Livingston, M.A.: Superior augmented reality registration by integrating landmark tracking and magnetic tracking. In: SIGGRAPH, pp. 429–438 (1996)
Lorensen, W.E., Cline, H.E., Nafis, C., Altobelli, D., Gleason, L.: Enhancing reality in the operating room. In: IEEE Visualization, pp. 410–415 (1993)
Rosenthal, M., State, A., Lee, J., Hirota, G., Ackerman, J., Keller, K., Pisano, E., Jiroutek, M., Muller, K., Fuchs, H.: Augmented reality guidance for needle biopsies: An initial randomized, controlled trial in phantoms. In: Medical Image Analysis, vol. 6, pp. 313–320. Elsevier, Amsterdam (2002)
Höllerer, T., Feiner, S., Terauchi, T., Rashid, G., Hallaway, D.: Exploring MARS: developing indoor and outdoor user interfaces to a mobile augmented reality system. In: Computers & Graphics, vol. 23, pp. 779–785. Elsevier, Amsterdam (1999)
Welch, G., Sonnenwald, D., Mayer-Patel, K., Yang, R., State, A., Towles, H., Cairns, B., Fuchs, H.: Remote 3D medical consultation. In: Proc. BROADMED Conf. 2005, University College of Borås. Swedish School of Library and Information Science, pp. 103–110 (2005)
Julier, S., Baillot, Y., Brown, D., Lanzagorta, M.: Information filtering for mobile augmented reality. IEEE Computer Graphics and Applications 22(5), 12–15 (2002)
Thomas, B., Piekarski, W., Hepworth, D., Gunther, B., Demczuk, V.: A wearable computer system with augmented reality to support terrestrial navigation. In: ISWC 1998: Proceedings of the 2nd IEEE International Symposium on Wearable Computers, Washington, DC, USA, p. 168. IEEE Computer Society, Los Alamitos (1998)
Feiner, S., MacIntyre, B., Hollerer, T., Webster, A.: A touring machine: Prototyping 3d mobile augmented reality systems for exploring the urban environment. In: ISWC 1997: Proceedings of the 1st IEEE International Symposium on Wearable Computers, Washington, DC, USA, p. 74. IEEE Computer Society, Los Alamitos (1997)
Rekimoto, J.: Navicam: A magnifying glass approach to augmented reality. In: Presence, vol. 6, pp. 399–412 (1997)
Thomas, B., Close, B., Donoghue, J., Squires, J., Bondi, P., Piekarski, W.: First Person Indoor/Outdoor Augmented Reality Application: ARQuake. In: Personal and Ubiquitous Computing, vol. 6, pp. 75–86. Springer, Heidelberg (2002)
Cheok, A., Goh, K., Liu, W., Farbiz, F., Fong, S., Teo, S., Li, Y., Yang, X.: Human Pacman: a mobile, wide-area entertainment system based on physical, social, and ubiquitous computing. In: Personal and Ubiquitous Computing, vol. 8, pp. 71–81. Springer, Heidelberg (2004)
Avery, B., Thomas, B., Velikovsky, J., Piekarski, W.: Outdoor augmented reality gaming on five dollars a day. In: Proceedings of the Sixth Australasian conference on User interface, vol. 40, pp. 79–88. Australian Computer Society, Inc, Darlinghurst (2005)
Szalavári, Z., Eckstein, E., Gervautz, M.: Collaborative gaming in augmented reality. In: Proceedings of the ACM symposium on Virtual reality software and technology 1998, pp. 195–204. ACM Press, New York (1998)
Barakonyi, I., Weilguny, M., Psik, T., Schmalstieg, D.: Monkeybridge: autonomous agents in augmented reality games. In: Lee, N. (ed.) Advances in Computer Entertainment Technology, pp. 172–175. ACM, New York (2005)
Bungert, C.: Hmd/headset/vr-helmet comparison chart (2008), http://www.stereo3d.com/hmd.htm
Rekimoto, J., Nagao, K.: The world through the computer: computer augmented interaction with real world environments. In: Proceedings of the 8th annual ACM symposium on User interface and software technology, pp. 29–36. ACM, New York (1995)
Moehring, M., Lessig, C., Bimber, O.: Video See-Through AR on Consumer Cell Phones. In: Proc. of International Symposium on Augmented and Mixed Reality (ISMAR 2004), pp. 252–253 (2004)
Olwal, A.: Lightsense: enabling spatially aware handheld interaction devices. In: ISMAR, pp. 119–122 (2006)
Wellner, P.: Interacting with paper on the DigitalDesk. In: Communications of the ACM, vol. 36, pp. 87–96. ACM Press, New York (1993)
Fitzmaurice, G., Ishii, H., Buxton, W.: Bricks: laying the foundations for graspable user interfaces. In: Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 442–449. ACM Press/Addison-Wesley Publishing Co., New York (1995)
Pinhanez, C.: Augmenting Reality with Projected Interactive Displays. In: Virtual and Augmented Architecture (Vaa 2001): Proceedings of the International Symposium on Virtual and Augmented Architecture (Vaa 2001), Trinity College, Dublin, June 21-22 (2001)
Olwal, A., Gustafsson, J., Lindfors, C.: Spatial augmented reality on industrial cnc-machines. In: Proceedings of SPIE 2008 Electronic Imaging. The Engineering Reality of Virtual Reality 2008, vol. 6804 (2008)
Raskar, R., van Baar, J., Beardsley, P., Willwacher, T., Rao, S., Forlines, C.: iLamps: geometrically aware and self-configuring projectors. In: International Conference on Computer Graphics and Interactive Techniques. ACM Press, New York (2006)
Getting, I.: Perspective/navigation-The Global Positioning System. In: Spectrum, vol. 30. IEEE, Los Alamitos (1993)
Steed, A.: Supporting Mobile Applications with Real-Time Visualisation of GPS Availability. In: Proceedings of Mobile HCI. Springer, Heidelberg (2004)
Kato, H., Billinghurst, M., Poupyrev, I., Imamoto, K., Tachibana, K.: Virtual object manipulation on a table-top AR environment. In: Proc. IEEE and ACM International Symposium on Augmented Reality (ISAR), pp. 111–119 (2000)
Costanza, E., Robinson, J.: A region adjacency tree approach to the detection and design of fiducials. In: Vision, Video and Graphics (VVG), pp. 63–70 (2003)
Lepetit, V., Fua, P.: Keypoint Recognition Using Randomized Trees. In: IEEE Transactions On Pattern Analysis And Machine Intelligence, pp. 1465–1479. IEEE Computer Society, Los Alamitos (2006)
Polhemus: Polhemus “fastrak” commercial system (retrieved, June 2008), http://www.polhemus.com/?page=Motion_Fastrak
Ascension: Ascension “flock of birds” commercial system (retrieved, June 2008), http://www.ascension-tech.com/products/flockofbirds.php
Foxlin, E., Harrington, M., Harrington, M., Pfeifer, G.: Constellation: a wide-range wireless motion-tracking system for augmented reality and virtual set applications. In: Proceedings of the 25th annual conference on Computer graphics and interactive techniques, pp. 371–378. ACM New York, NY (1998)
Yokokohji, Y., Sugawara, Y., Yoshikawa, T.: Accurate image overlay on video see-through hmds using vision and accelerometers. In: VR 2000: Proceedings of the IEEE Virtual Reality 2000 Conference, Washington, DC, USA, p. 247. IEEE Computer Society, Los Alamitos (2000)
Kotake, D., Satoh, K., Uchiyama, S., Yamamoto, H.: A hybrid and linear registration method utilizing inclination constraint. In: Proceedings of the Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 140–149 (2005)
Azuma, R., Bishop, G.: A frequency-domain analysis of head-motion prediction. In: SIGGRAPH 1995: Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, pp. 401–408. ACM, New York (1995)
Dourish, P.: Where the action is: the foundations of embodied interaction. MIT Press, Cambridge (2001)
Swan, J.E., Gabbard, J.L.: Survey of user-based experimentation in augmented reality. In: Proceedings of 1st International Conference on Virtual Reality (2005)
Fjeld, M.: Special Issue on Usability and collaborative aspects of augmented reality. Interactions 11(6), 11–15 (2004)
Billinghurst, M., Belcher, D., Gupta, A., Kiyokawa, K.: Communication Behaviors in Colocated Collaborative AR Interfaces. International Journal of Human-Computer Interaction 16(3), 395–423 (2003)
Wiedenmaier, S., Oehme, O., Schmidt, L., Luczak, H.: Augmented reality (ar) for assembly processes design and experimental evaluation. Int. J. Hum. Comput. Interaction 16(3), 497–514 (2003)
Papagiannakis, G., Schertenleib, S., Ponder, M., Arévalo-Poizat, M., Magnenat-Thalmann, N., Thalmann, D.: Real-time virtual humans in ar sites. In: 1st European Conference on Visual Media Production (CVMP), pp. 273–276. IEEE Publisher, Los Alamitos (2004)
Bianchi, G., Knörlein, B., Szèkely, G., Harders, M.: High precision augmented reality haptics. In: Eurohaptics (2006)
Bianchi, G., Wengert, C., Harders, M., Cattin, P., Székely, G.: Camera-marker alignment framework and comparison with hand-eye calibration for augmented reality applications. In: ISMAR (2005)
Fjeld, M., Lauche, K., Bichsel, M., Voorhorst, F., Krueger, H., Rauterberg, M.: Physical and virtual tools: Activity theory applied to the design of groupware. In: Computer Supported Cooperative Work (CSCW), vol. 11, pp. 153–180 (2002)
Fjeld, M., Morf, M., Krueger, H.: Activity theory and the practice of design: evaluation of a collaborative tangible user interface. Inderscience 4, 94–116 (2004)
Ganser Schwab, C., Kennel, T., Kunz, A.: Digital support for net-based teamwork in early design stages. Journal of Design Research 6(1), 150–168 (2007)
Schwab, C.G., Kennel, T., Kunz, A.: Digital support for net-based teamwork in early design stages 6(1/2), 150–168 (2007)
Gross, M.H., Würmlin, S., Näf, M., Lamboray, E., Spagno, C.P., Kunz, A.M., Koller-Meier, E., Svoboda, T., Gool, L.J.V., Lang, S., Strehlke, K., Moere, A.V., Staadt, O.G.: Blue-c: a spatially immersive display and 3d video portal for telepresence. ACM Trans. Graph. 22(3), 819–827 (2003)
Kuechler, M., Kunz, A.: Holoport - a device for simultaneous video and data conferencing featuring gaze awareness. In: Proceedings of the IEEE conference on Virtual Reality (IEEE VR 2006), Washington, DC, United States, pp. 81–87. IEEE Computer Society, Los Alamitos (2006)
Kunz, A., Fadel, G., Taiber, J., Schichtel, M.: Towards collaboration in engineering of tomorrow - building highly interactive virtual collaboration platforms. In: SAE 2006 World Congress & Exhibition, United States, SAE International (2006)
Kato, H., Billinghurst, M.: Marker tracking and hmd calibration for a video-based augmented reality conferencing system. In: IWAR 1999: Proceedings of the 2nd IEEE and ACM International Workshop on Augmented Reality. IEEE Computer Society, Los Alamitos (1999)
Voegtli, B.: Augmented collaboration. M.sc. thesis, HyperWerk FHNW (2002)
Fjeld, M., Fredriksson, J., Ejdestig, M., Duca, F., Bötschi, K., Voegtli, B.M., Juchli, P.: Tangible user interface for chemistry education: comparative evaluation and re-design. In: CHI, pp. 805–808 (2007)
Iten, C., Lüthi, D.: (2006), http://www.zima.ch/tangent/
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Costanza, E., Kunz, A., Fjeld, M. (2009). Mixed Reality: A Survey. In: Lalanne, D., Kohlas, J. (eds) Human Machine Interaction. Lecture Notes in Computer Science, vol 5440. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-00437-7_3
Download citation
DOI: https://doi.org/10.1007/978-3-642-00437-7_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-00436-0
Online ISBN: 978-3-642-00437-7
eBook Packages: Computer ScienceComputer Science (R0)