Abstract
With vision being a primary sense of humans, we often first estimate the physical properties of objects by looking at them. However, when in doubt, for example, about the material they are made of or its structure, it is natural to apply other senses, such as haptics by touching them. Aiming at the ultimate goal of achieving a full-sensory augmented reality experience, we present an initial study focusing on multimodal feedback when tapping an object to estimate the thickness of its material. Our results indicate that we can change the perception of thickness of stiff objects by modulating acoustic stimuli. For flexible objects, which have a more distinctive tactile characteristic, adding vibratory responses when tapping on thick objects can make people perceive them as thin. We also identified that in the latter case, adding congruent acoustic stimuli does not further enhance the illusion but worsens it.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Avanzini, F., Crosato, P.: Integrating physically based sound models in a multimodal rendering architecture. Comput. Anim. Virtual Worlds 17(3–4), 411–419 (2006)
Charpentier, A.: Experimental study of some aspects of weight perception. Archives de Physiol. Norm. Pathol. 3, 122–135 (1981)
DiFranco, D.E., Beauregard, G.L., Srinivasan, M.A.: The effect of auditory cues on the haptic perception of stiffness in virtual environments. In: Proceedings of the ASME Dynamic Systems and Control Division, vol. 61, pp. 17–22 (1997)
Guest, S., Catmur, C., Lloyd, D., Spence, C.: Audiotactile interactions in roughness perception. Exp. Brain Res. 146(2), 161–171 (2002)
Hachisu, T., Sato, M., Fukushima, S., Kajimoto, H.: Augmentation of material property by modulating vibration resulting from tapping. In: Proceedings of the International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, pp. 173–180 (2012)
Hirano, Y., Kimura, A., Shibata, F., Tamura, H.: Psychophysical influence of mixed-reality visual stimulation on sense of hardness. In: Proceedings of IEEE Virtual Reality, pp. 51–54 (2011)
Iesaki, A., Somada, A., Kimura, A., Shibata, F., Tamura, H.: Psychophysical influence on tactual impression by mixed-reality visual stimulation. In: 2008 IEEE Virtual Reality Conference, VR 2008, pp. 265–266. IEEE (2008)
Kagimoto, M., Kimura, A., Shibata, F., Tamura, H.: Analysis of tactual impression by audio and visual stimulation for user interface design in mixed reality environment. In: Proceedings of the International Conference on Virtual and Mixed Reality, pp. 326–335 (2009)
Landau, L.D., Lifshitz, E.: Theory of Elasticity, vol. 1. Course of Theoretical Physics, vol. 3, p. 109 (1986)
Lederman, S.J., Klatzky, R.L., Morgan, T., Hamilton, C.: Integrating multimodal information about surface texture via a probe: relative contributions of haptic and touch-produced sound sources. In: 2002 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, HAPTICS 2002, Proceedings, pp. 97–104. IEEE (2002)
Minamizawa, K., Kakehi, Y., Nakatani, M., Mihara, S., Tachi, S.: Techtile toolkit: a prototyping tool for design and education of haptic media. In: Proceedings of the 2012 Virtual Reality International Conference, p. 26. ACM (2012)
Mogees Ltd. Mogees pro (2016). http://www.mogees.co.uk/pro. Accessed 15 Sept 2017
Narumi, T., Nishizaka, S., Kajinami, T., Tanikawa, T., Hirose, M.: Augmented reality flavors: gustatory display based on edible marker and cross-modal interaction. In: Proceedings of Human Factors in Computing Systems, pp. 93–102. ACM (2011)
Punpongsanon, P., Iwai, D., Sato, K.: SoftAR: visually manipulating haptic softness perception in spatial augmented reality. IEEE Trans. Vis. Comput. Graph. 21(11), 1279–1288 (2015)
Rosa, N., Hürst, W., Vos, W., Werkhoven, P.: The Influence of visual cues on passive tactile sensations in a multimodal immersive virtual environment. In: Proceedings of the ACM International Conference on Multimodal Interaction, pp. 327–334 (2015)
Van Den Doel, K., Kry, P.G., Pai, D.K.: FoleyAutomatic: physically-based sound effects for interactive simulation and animation. In: Proceedings of ACM Computer Graphics and Interactive Techniques, pp. 537–544 (2001)
Wu, W.-C., Basdogan, C., Srinivasan, M.A.: Visual, haptic, and bimodal perception of size and stiffness in virtual environments. ASME Dyn. Syst. Control Div. Publ. DSC 67, 19–26 (1999)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this paper
Cite this paper
Lugtenberg, G. et al. (2018). Multimodal Augmented Reality – Augmenting Auditory-Tactile Feedback to Change the Perception of Thickness. In: Schoeffmann, K., et al. MultiMedia Modeling. MMM 2018. Lecture Notes in Computer Science(), vol 10704. Springer, Cham. https://doi.org/10.1007/978-3-319-73603-7_30
Download citation
DOI: https://doi.org/10.1007/978-3-319-73603-7_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-73602-0
Online ISBN: 978-3-319-73603-7
eBook Packages: Computer ScienceComputer Science (R0)