research-article

Mixed reality based interaction system for digital heritage

Authors:

Nishant Bugalia,

Shantanu ChoudharyAuthors Info & Claims

VRCAI '16: Proceedings of the 15th ACM SIGGRAPH Conference on Virtual-Reality Continuum and Its Applications in Industry - Volume 1

Pages 31 - 37

https://doi.org/10.1145/3013971.3014000

Published: 03 December 2016 Publication History

Abstract

User interfaces that leverages vivid computer graphics and virtual reality technologies provide effective means for exploring and inspecting cultural artifacts. Such virtual inspection systems help experience and disseminate cultural heritage. For such systems to be truly ubiquitous, they need to expose a user-friendly interface that an untrained population can find intuitive and engaging. In this paper, we present one such mixed reality (MR) based interaction system that can be used to visualize as well as explore cultural heritage. The system is demonstrated by an interactive exploration of 15th century 'Vittala Temple' located at a UNESCO world heritage site, Hampi. Our system investigates the augmentation of computer generated imagery with a physical replica to increase immersion and awareness. It uses monoscopic projection mapping on the 3D-printed replica laid out on a tabletop along with a vertical projection wall behind the tabletop, on which a 3D-rendered view is displayed. Projection on the physical layout is synchronized with the 3D rendering to allow projection of texture, navigation path, users current position, etc. on the tabletop. Users can interact with this system by simply waving a laser pointer to select structures or to define a path. We have additionally explored the benefit of using a steering wheel to further control the camera direction. Such a system creates and maintains a semantic relation between users actions and the resulting effect. The UI enables two major functionalities: interactive inspection of an architectural complex and the multimedia data attached to various components and artifacts.

Supplementary Material

ZIP File (p31-bugalia.zip)

Supplemental material.

Download
21.18 MB

References

[1]

Andujar, C., Chica, A., and Brunet, P. 2012. User-interface design for the ripoll monastery exhibition at the national art museum of catalonia. Computers & Graphics 36, 1, 28--37.

Digital Library

[2]

Anguelov, D., Dulong, C., Filip, D., Frueh, C., Lafon, S., Lyon, R., Ogale, A., Vincent, L., and Weaver, J. 2010. Google street view: Capturing the world at street level.

[3]

Baudisch, P., Good, N., and Stewart, P. 2001. Focus plus context screens: combining display technology with visualization techniques. In Proceedings of the 14th annual ACM symposium on User interface software and technology, ACM, 31--40.

Digital Library

[4]

Bettio, F., Gobbetti, E., Marton, F., Tinti, A., Merella, E., and Combet, R. 2009. A point-based system for local and remote exploration of dense 3d scanned models. In VAST, Citeseer, 25--32.

Digital Library

[5]

Bonanni, L., Vargas, G., Chao, N., Pueblo, S., and Ishii, H. 2009. Spime builder: a tangible interface for designing hyperlinked objects. In Proceedings of the 3rd International Conference on Tangible and Embedded Interaction, ACM, 263--266.

Digital Library

[6]

Bruder, G., Steinicke, F., and Hinrichs, K. H. 2009. Archexplore: A natural user interface for immersive architectural walkthroughs. In 3D User Interfaces, 2009. 3DUI 2009. IEEE Symposium on, IEEE, 75--82.

Digital Library

[7]

Burigat, S., and Chittaro, L. 2007. Navigation in 3d virtual environments: Effects of user experience and location-pointing navigation aids. Int. J. Hum.-Comput. Stud. 65, 11 (Nov.), 945--958.

Digital Library

[8]

Chittaro, L., and Venkataraman, S. 2006. Navigation aids for multi-floor virtual buildings: A comparative evaluation of two approaches. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology, ACM, New York, NY, USA, VRST '06, 227--235.

Digital Library

[9]

Computer program to calculate kendall w. http://www.statstodo.com/KendallW_Pgm.php. (Accessed on 07/09/2016).

[10]

Cruz-Neira, C., Sandin, D. J., DeFanti, T. A., Kenyon, R. V., and Hart, J. C. 1992. The cave: audio visual experience automatic virtual environment. Communications of the ACM 35, 6, 64--73.

Digital Library

[11]

Darken, R. P., and Sibert, J. L. 1993. A toolset for navigation in virtual environments. In Proceedings of the 6th Annual ACM Symposium on User Interface Software and Technology, ACM, New York, NY, USA, UIST '93, 157--165.

Digital Library

[12]

Fjeld, M., Bichsel, M., and Rauterberg, M. 1999. Buildit: a brick-based tool for direct interaction. Engineering Psychology and Cognitive Ergonomics (EPCE) 4, 205--212.

[13]

Fowler, R. H., Carrillo, M. D., Huerta, R. A., and Fowler, W. A. 2012. Designing a low cost immersive environment system twenty years after the first cave. In Proceedings of the International Conference on Computer Graphics and Virtual Reality (CGVR), The Steering Committee of The World Congress in Computer Science, Computer Engineering and Applied Computing (WorldComp), 1.

[14]

Indian digital heritage. http://www.nrdms.gov.in/idh.asp. (Accessed on 09/24/2016).

[15]

Igarashi, T., Kadobayashi, R., Mase, K., and Tanaka, H. 1998. Path drawing for 3d walkthrough. In Proceedings of the 11th Annual ACM Symposium on User Interface Software and Technology, ACM, New York, NY, USA, UIST '98, 173--174.

Digital Library

[16]

IMU, R. 9 degrees of freedom - razor imu - sen-10736 - sparkfun electronics. https://www.sparkfun.com/products/10736. (Accessed on 07/09/2016).

[17]

Inertial measurement unit - wikipedia, the free encyclopedia. https://en.wikipedia.org/wiki/Inertial_measurement_unit. (Accessed on 07/09/2016).

[18]

Juarez, A., Schonenberg, W., and Bartneck, C. 2010. Implementing a low-cost cave system using the cryengine2. Entertainment Computing 1, 3, 157--164.

[19]

Kendall's w - wikipedia, the free encyclopedia. https://en.wikipedia.org/wiki/Kendall%27s_W. (Accessed on 07/09/2016).

[20]

Müller-Tomfelde, C. 2010. Tabletops-Horizontal Interactive Displays. Springer Science & Business Media.

Digital Library

[21]

Nielsen, J., and Molich, R. 1990. Heuristic evaluation of user interfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems, ACM, 249--256.

Digital Library

[22]

Objet24 desktop 3d printer --- stratasys. http://www.stratasys.com/3d-printers/design-series/objet24. (Accessed on 06/02/2016).

[23]

Oculus, V. 2015. Oculus rift. Available from WWW:¡ http://www.oculusvr.com/rift.

[24]

Photosynth. Photosynth - capture your world in 3d. https://photosynth.net/. (Accessed on 05/16/2016).

[25]

Piazza, T., and Fjeld, M. 2007. Ortholumen: Using light for direct tabletop input. In Horizontal Interactive Human-Computer Systems, 2007. TABLETOP'07. Second Annual IEEE International Workshop on, IEEE, 193--196.

[26]

Piazza, T., Heller, H., and Fjeld, M. 2009. Cermit: Co-located and remote collaborative system for emergency response management. Proc. SIGRAD, 12--20.

[27]

project morpheus, V. Project morpheus - playstation. https://www.playstation.com/en-us/explore/project-morpheus/. (Accessed on 05/16/2016).

[28]

Science express climate action special. http://www.sciencexpress.in/about-science-express.html. (Accessed on 05/16/2016).

[29]

Sjölinder, M., Höök, K., Nilsson, L.-G., and Andersson, G. 2005. Age differences and the acquisition of spatial knowledge in a three-dimensional environment: Evaluating the use of an overview map as a navigation aid. Int. J. Hum.-Comput. Stud. 63, 6 (Dec.), 537--564.

Digital Library

[30]

Stoakley, R., Conway, M. J., and Pausch, R. 1995. Virtual reality on a wim: Interactive worlds in miniature. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, CHI '95, 265--272.

Digital Library

[31]

Ullmer, B., and Ishii, H. 1997. The metadesk: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software and technology, ACM, 223--232.

Digital Library

[32]

Underkoffler, J., and Ishii, H. 1999. Urp: a luminous-tangible workbench for urban planning and design. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems, ACM, 386--393.

Digital Library

[33]

Unity - game engine. http://unity3d.com/. (Accessed on 05/16/2016).

[34]

Vittala temple. http://hampi.in/vittala-temple. (Accessed on 05/16/2016).

Cited By

Karre SReddy YMittal R(2024)RE Methods for Virtual Reality Software Product Development: A Mapping StudyACM Transactions on Software Engineering and Methodology10.1145/364959533:4(1-31)Online publication date: 26-Feb-2024
https://dl.acm.org/doi/10.1145/3649595
Tongpaeng YNobnop RWongwan NHomla PIntawong KPuritat K(2024)Comparison of gamified and non-gamified mixed reality in enhancing museum visitor engagement, motivation, and learning outcomeJournal of Heritage Tourism10.1080/1743873X.2024.2351852(1-30)Online publication date: 16-May-2024
https://doi.org/10.1080/1743873X.2024.2351852
Papagiannakis GGeronikolakis EPateraki MLópez-Menchero VTsioumas MSylaiou SLiarokapis FGrammatikopoulou ADimitropoulos KGrammalidis NPartarakis NMargetis GDrossis GVassiliadi MChalmers AStephanidis CMagnenat-Thalmann N(2024)Mixed Reality, Gamified Presence, and Storytelling for Virtual MuseumsEncyclopedia of Computer Graphics and Games10.1007/978-3-031-23161-2_249(1150-1162)Online publication date: 5-Jan-2024
https://doi.org/10.1007/978-3-031-23161-2_249
Show More Cited By

Index Terms

Mixed reality based interaction system for digital heritage
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Graphics input devices

Recommendations

A Review on Augmented Reality for Virtual Heritage System
Edutainment '09: Proceedings of the 4th International Conference on E-Learning and Games: Learning by Playing. Game-based Education System Design and Development

Augmented reality is one of the emerging technologies to reconstruct the historical building and monument in the previous era, where the user experiences with the real environment or virtual scene. In education, Virtual Heritage becomes as a platform of ...
Capturing aboriginal heritage in virtual reality
VRST '17: Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology

The culture of Aboriginal Australians is unique and diverse, but, unfortunately, it is under threat of extinction. This is the culture that does not rely on written records, as knowledge is predominantly being shared in the form of oral tales and ...
5D GIS virtual heritage

In recent decade, a wave of apprehension swept across the Pearl of Oriental with regards to the disappearance of high value tangible and intangible heritages due to urban renewal and redevelopment. The flourish of computer science and engineering, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

VRCAI '16: Proceedings of the 15th ACM SIGGRAPH Conference on Virtual-Reality Continuum and Its Applications in Industry - Volume 1

December 2016

381 pages

ISBN:9781450346924

DOI:10.1145/3013971

Conference Chairs:
Yiyu Cai
Nanyang Technological University, Singapore
,
Daniel Thalmann
École Polytechnique Fédérale de Lausanne, Switzerland and Nanyang Technological University, Singapore

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 December 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

VRCAI '16

Sponsor:

SIGGRAPH

VRCAI '16: The 15th International Conference on Virtual-Reality Continuum and its Applications in Industry

December 3 - 4, 2016

Zhuhai, China

Acceptance Rates

Overall Acceptance Rate 51 of 107 submissions, 48%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
376
Total Downloads

Downloads (Last 12 months)42
Downloads (Last 6 weeks)1

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Karre SReddy YMittal R(2024)RE Methods for Virtual Reality Software Product Development: A Mapping StudyACM Transactions on Software Engineering and Methodology10.1145/364959533:4(1-31)Online publication date: 26-Feb-2024
https://dl.acm.org/doi/10.1145/3649595
Tongpaeng YNobnop RWongwan NHomla PIntawong KPuritat K(2024)Comparison of gamified and non-gamified mixed reality in enhancing museum visitor engagement, motivation, and learning outcomeJournal of Heritage Tourism10.1080/1743873X.2024.2351852(1-30)Online publication date: 16-May-2024
https://doi.org/10.1080/1743873X.2024.2351852
Papagiannakis GGeronikolakis EPateraki MLópez-Menchero VTsioumas MSylaiou SLiarokapis FGrammatikopoulou ADimitropoulos KGrammalidis NPartarakis NMargetis GDrossis GVassiliadi MChalmers AStephanidis CMagnenat-Thalmann N(2024)Mixed Reality, Gamified Presence, and Storytelling for Virtual MuseumsEncyclopedia of Computer Graphics and Games10.1007/978-3-031-23161-2_249(1150-1162)Online publication date: 5-Jan-2024
https://doi.org/10.1007/978-3-031-23161-2_249
Li YDu YYang MLiang JBai HLi RLaw A(2023)A review of the tools and techniques used in the digital preservation of architectural heritage within disaster cyclesHeritage Science10.1186/s40494-023-01035-x11:1Online publication date: 19-Sep-2023
https://doi.org/10.1186/s40494-023-01035-x
Olaz XGarcia ROrtiz AMarichal SVilladangos JArdaiz OMarzo A(2022)An Interdisciplinary Design of an Interactive Cultural Heritage Visit for In-Situ, Mixed Reality and Affective ExperiencesMultimodal Technologies and Interaction10.3390/mti60700596:7(59)Online publication date: 18-Jul-2022
https://doi.org/10.3390/mti6070059
Kontopanagou KTsipis AKomianos V(2021)A Framework for Exploring Churches/Monuments/Museums of Byzantine Cultural Influence Exploiting Immersive Technologies in Real-Time Networked EnvironmentsTechnologies10.3390/technologies90300579:3(57)Online publication date: 9-Aug-2021
https://doi.org/10.3390/technologies9030057
Marto AMelo MGoncalves ABessa M(2021)Development and Evaluation of an Outdoor Multisensory AR System for Cultural HeritageIEEE Access10.1109/ACCESS.2021.30509749(16419-16434)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3050974
Marto AGoncalves ABessa M(2019)SensiMAR - Designing a Multisensory AR System2019 International Conference on Graphics and Interaction (ICGI)10.1109/ICGI47575.2019.8955078(9-16)Online publication date: Nov-2019
https://doi.org/10.1109/ICGI47575.2019.8955078
Papagiannakis GGeronikolakis EPateraki MLópez-Menchero VTsioumas MSylaiou SLiarokapis FGrammatikopoulou ADimitropoulos KGrammalidis NPartarakis NMargetis GDrossis GVassiliadi MChalmers AStephanidis CMagnenat-Thalmann N(2019)Mixed Reality, Gamified Presence, and Storytelling for Virtual MuseumsEncyclopedia of Computer Graphics and Games10.1007/978-3-319-08234-9_249-2(1-13)Online publication date: 9-Apr-2019
https://doi.org/10.1007/978-3-319-08234-9_249-2
Scott MParker APowley ESaunders RLee JHerring PBrown DKrzywinska T(2018)Towards an interaction blueprint for mixed reality experiences in GLAM spacesProceedings of the 32nd International BCS Human Computer Interaction Conference10.14236/ewic/HCI2018.135(1-5)Online publication date: 4-Jul-2018
https://dl.acm.org/doi/10.14236/ewic/HCI2018.135
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten