Abstract
In this paper, we present a multi-sensory perception consistent 6-DOF motion system. The system automatically extracts the motion trajectory of the virtual camera as motion data from video and maps the motion data to the 6-DOF Stewart motion platform through a human perception-based wash-out algorithm and incorporates multi-sensory simulations of visual, auditory, tactile, and proprioceptive sensory perceptual consistency of the motion effect. The results of the user study showed that the system effectively enhanced the participants’ sense of realism and reduced the subjective perception of simulator discomfort. In addition, the system well supported users to self-create motion virtual environment through video, so that the public became the designer of motion experience content in the metaverse.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Adel, A., et al.: Design of a 6-DOF hydraulic vehicle driving simulator. In: 2020 International Conference on Innovative Trends in Communication and Computer Engineering (ITCE), pp. 170–175. IEEE (2020)
Asadi, H., Bellmann, T., Mohamed, S., Lim, C.P., Khosravi, A., Nahavandi, S.: Adaptive motion cueing algorithm using optimized fuzzy control system for motion simulators. IEEE Trans. Intell. Veh. 8, 390–403 (2022)
Asadi, H., Lim, C.P., Mohamed, S., Nahavandi, D., Nahavandi, S.: Increasing motion fidelity in driving simulators using a fuzzy-based washout filter. IEEE Trans. Intell. Veh. 4(2), 298–308 (2019)
Bimberg, P., Weissker, T., Kulik, A.: On the usage of the simulator sickness questionnaire for virtual reality research. In: 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), pp. 464–467. IEEE (2020)
Buzan, D., Sclaroff, S., Kollios, G.: Extraction and clustering of motion trajectories in video. In: Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004, vol. 2, pp. 521–524. IEEE (2004)
Campos, C., Elvira, R., Rodríguez, J.J.G., Montiel, J.M., Tardós, J.D.: ORB-SLAM3: an accurate open-source library for visual, visual-inertial, and multimap slam. IEEE Trans. Robot. 37(6), 1874–1890 (2021)
Clifton, J., Palmisano, S.: Effects of steering locomotion and teleporting on cybersickness and presence in HMD-based virtual reality. Virtual Reality 24(3), 453–468 (2020)
Dinh, H.Q., Walker, N., Hodges, L.F., Song, C., Kobayashi, A.: Evaluating the importance of multi-sensory input on memory and the sense of presence in virtual environments. In: Proceedings of the IEEE Virtual Reality (Cat. No. 99CB36316), pp. 222–228. IEEE (1999)
Feng, M., Dey, A., Lindeman, R.W.: The effect of multi-sensory cues on performance and experience during walking in immersive virtual environments. In: 2016 IEEE Virtual Reality (VR), pp. 173–174. IEEE (2016)
Feng, M., Dey, A., Lindeman, R.W.: An initial exploration of a multi-sensory design space: tactile support for walking in immersive virtual environments. In: 2016 IEEE Symposium on 3D User Interfaces (3DUI), pp. 95–104. IEEE (2016)
Hawkins, D.G.: Virtual reality and passive simulators: the future of fun. Commun. Age Virtual Reality 1, 159–89 (1995)
Kaliuzhna, M., Ferrè, E.R., Herbelin, B., Blanke, O., Haggard, P.: Multisensory effects on somatosensation: a trimodal visuo-vestibular-tactile interaction. Sci. Rep. 6(1), 26301 (2016)
Khusro, Y.R., Zheng, Y., Grottoli, M., Shyrokau, B.: MPC-based motion-cueing algorithm for a 6-DOF driving simulator with actuator constraints. Vehicles 2(4), 625–647 (2020)
Lee, J., Han, B., Choi, S.: Motion effects synthesis for 4d films. IEEE Trans. Vis. Comput. Graph. 22(10), 2300–2314 (2015)
Melo, M., Gonçalves, G., Monteiro, P., Coelho, H., Vasconcelos-Raposo, J., Bessa, M.: Do multisensory stimuli benefit the virtual reality experience? A systematic review. IEEE Trans. Vis. Comput. Graph. 28(2), 1428–1442 (2020)
Nehaoua, L., Mohellebi, H., Amouri, A., Arioui, H., Espié, S., Kheddar, A.: Design and control of a small-clearance driving simulator. IEEE Trans. Veh. Technol. 57(2), 736–746 (2008)
Qazani, M.R.C., Asadi, H., Bellmann, T., Mohamed, S., Lim, C.P., Nahavandi, S.: Adaptive washout filter based on fuzzy logic for a motion simulation platform with consideration of joints’ limitations. IEEE Trans. Veh. Technol. 69(11), 12547–12558 (2020)
Qazani, M.R.C., Asadi, H., Nahavandi, S.: An optimal motion cueing algorithm using the inverse kinematic solution of the hexapod simulation platform. IEEE Trans. Intell. Veh. 7(1), 73–82 (2021)
Ranasinghe, N., et al.: Season traveller: multisensory narration for enhancing the virtual reality experience. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1–13 (2018)
Rheinberg, F., Engeser, S., Vollmeyer, R.: Measuring components of flow: the flow-short-scale. In: Proceedings of the 1st International Positive Psychology Summit (2002)
Seo, S.M., Kimm, M.J.: Analysis of Virtual Reality Movies: Focusing on the Effect of Virtual Reality Movie’s Distinction on User Experience, pp. 308–312, July 2023. https://doi.org/10.1007/978-3-031-36004-6_42
Sharma, A., Sharma, S., Chaudhary, M.: Are small travel agencies ready for digital marketing? Views of travel agency managers. Tour. Manag. 79, 104078 (2020)
Sheng, B., Li, P., Ali, R., Chen, C.L.P.: Improving video temporal consistency via broad learning system. IEEE Trans. Cybern. 52(7), 6662–6675 (2022). https://doi.org/10.1109/TCYB.2021.3079311
Shin, S., Yoo, B., Han, S.: A framework for automatic creation of motion effects from theatrical motion pictures. Multimedia Syst. 20, 327–346 (2014)
Stewart, D.: A platform with six degrees of freedom. Proc. Inst. Mech. Eng. 180(1), 371–386 (1965)
Wang, Y., Sun, X., Shen, H., Yin, Y.: Research on improvement and optimization of washout algorithm for moving platform navigation simulator. In: 2021 IEEE 7th International Conference on Virtual Reality (ICVR), pp. 400–406. IEEE (2021)
Yang, T., Lai, I.K.W., Fan, Z.B., Mo, Q.M.: The impact of a 360 virtual tour on the reduction of psychological stress caused by COVID-19. Technol. Soc. 64, 101514 (2021)
Yun, G., Lee, H., Han, S., Choi, S.: Improving viewing experiences of first-person shooter gameplays with automatically-generated motion effects. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1–14 (2021)
Zachmann, G., Alcañiz Raya, M., Bourdot, P., Marchal, M., Stefanucci, J., Yang, X.: Correction to: virtual reality and mixed reality. In: Zachmann, G., Alcaniz Raya, M., Bourdot, P., Marchal, M., Stefanucci, J., Yang, X. (eds.) Virtual Reality and Mixed Reality. EuroXR 2022. LNCS, vol. 13484, p. C1. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-16234-3_14
Zhu, J., et al.: Animating turbulent fluid with a robust and efficient high-order advection method. Comput. Animat. Virtual Worlds 31(4–5), e1951 (2020)
Zou, X., et al.: On-road virtual reality autonomous vehicle (VRAV) simulator: an empirical study on user experience. Transp. Res. Part C Emerg. Technol. 126, 103090 (2021)
Acknowledgements
We would like to thank all reviewers for their valuable comments. This work is supported by the National Natural Science Foundation of China under Grant (61972233, 62007021, 62277035).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Luan, H. et al. (2024). Multi-sensory Consistency Experience: A 6-DOF Simulation System Based on Video Automatically Generated Motion Effects. In: Sheng, B., Bi, L., Kim, J., Magnenat-Thalmann, N., Thalmann, D. (eds) Advances in Computer Graphics. CGI 2023. Lecture Notes in Computer Science, vol 14497. Springer, Cham. https://doi.org/10.1007/978-3-031-50075-6_36
Download citation
DOI: https://doi.org/10.1007/978-3-031-50075-6_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-50074-9
Online ISBN: 978-3-031-50075-6
eBook Packages: Computer ScienceComputer Science (R0)