Abstract
Virtual reality (VR) has garnered much interest as a training environment for motor skill acquisition, including for neurological rehabilitation of upper extremities. While the focus has been on gross upper limb motion, VR applications that involve reaching for, and interacting with, virtual objects are growing. The absence of true haptics in VR when it comes to hand-object interactions raises a fundamentally important question: can haptic-free immersive virtual environments (hf-VEs) support naturalistic coordination of reach-to-grasp movements? This issue has been grossly understudied, and yet is of significant importance in the development and application of VR across a number of sectors. In a previous study (Furmanek et al., J Neuroeng Rehabil 16:78, 2019), we reported that reach-to-grasp movements are similarly coordinated in both the physical environment (PE) and hf-VE. The most noteworthy difference was that the closure phase—which begins at maximum aperture and lasts through the end of the movement—was longer in hf-VE than in PE, suggesting that different control laws might govern the initiation of closure between the two environments. To do so, we reanalyzed data from Furmanek et al. (J Neuroeng Rehabil 16:78, 2019), in which the participants reached to grasp three differently sized physical objects, and matching 3D virtual object renderings, placed at three different locations. Our analysis revealed two key findings pertaining to the initiation of closure in PE and hf-VE. First, the respective control laws governing the initiation of aperture closure in PE and hf-VE both included state estimates of transport velocity and acceleration, supporting a general unified control policy for implementing reach-to-grasp across physical and virtual environments. Second, the aperture was less informative to the control law in hf-VE. We suggest that the latter was likely because transport velocity at closure onset and aperture at closure onset were less independent in hf-VE than in PE, ultimately resulting in an aperture at closure onset having a weaker influence on the initiation of closure. In this way, the excess time and muscular effort needed to actively bring the fingers to a stop at the interface of a virtual object was factored into the control law governing the initiation of closure in hf-VE. Critically, this control law remained applicable, albeit with different weights in hf-VE, despite the absence of terminal haptic feedback and potential perceptual differences.
Similar content being viewed by others
References
Adamovich SV, Fluet GG, Eugene T, Merians AS (2009) Sensorimotor training in virtual reality: a review. NeuroRehabilitation 25:25–44. https://doi.org/10.3233/NRE-2009-0497
Armbrüster C, Wolter M, Kuhlen T et al (2008) Depth perception in virtual reality: distance estimations in peri- and extrapersonal space. Cyberpsychol Behav 11:9–15. https://doi.org/10.1089/cpb.2007.9935
Bingham G, Coats R, Mon-Williams M (2007) Natural prehension in trials without haptic feedback but only when calibration is allowed. Neuropsychologia 45:288–294. https://doi.org/10.1016/j.neuropsychologia.2006.07.011
Bootsma RJ, Marteniuk RG, MacKenzie CL, Zaal FTJM (1994) The speed-accuracy trade-off in manual prehension: effects of movement amplitude, object size and object width on kinematic characteristics. Exp Brain Res 98:535–541. https://doi.org/10.1007/BF00233990
Borst CW, Volz RA (2005) Evaluation of a haptic mixed reality system for interactions with a virtual control panel. Presence Teleoperators Virtual Environ 14:677–696. https://doi.org/10.1162/105474605775196562
Burnham KP, Anderson DR (2002) Model selection and multimodel inference: a practical information-theoretic approach. Springer-Verlag, New York
Cavina-Pratesi C, Hesse C (2013) Why do the eyes prefer the index finger? Simultaneous recording of eye and hand movements during precision grasping. J Vis 13:15. https://doi.org/10.1167/13.5.15
Cheung KL, Tunik E, Adamovich SV, Boyd LA (2014) Neuroplasticity and virtual reality. In: Weiss (Tamar) PL, Keshner EA, Levin MF (eds) Virtual reality for physical and motor rehabilitation. Springer, New York, pp 5–24
Coats R, Bingham GP, Mon-Williams M (2008) Calibrating grasp size and reach distance: interactions reveal integral organization of reaching-to-grasp movements. Exp Brain Res 189:211–220. https://doi.org/10.1007/s00221-008-1418-5
Culbertson H, Schorr SB, Okamura AM (2018) Haptics: the present and future of artificial touch sensation. Annu Rev Control Robot Auton Syst 1:385–409. https://doi.org/10.1146/annurev-control-060117-105043
Curtin F, Schulz P (1998) Multiple correlations and Bonferroni’s correction. Biol Psychiatry 44:775–777. https://doi.org/10.1016/S0006-3223(98)00043-2
Fitts PM (1954) The information capacity of the human motor system in controlling the amplitude of movement. J Exp Psychol 47:381–391. https://doi.org/10.1037/h0055392
Fukui T, Inui T (2013) How vision affects kinematic properties of pantomimed prehension movements. Front Psychol 4:44. https://doi.org/10.3389/fpsyg.2013.00044
Furmanek MP, Schettino LF, Yarossi M et al (2019) Coordination of reach-to-grasp in physical and haptic-free virtual environments. J Neuroeng Rehabil 16:78. https://doi.org/10.1186/s12984-019-0525-9
Geiger A, Bewersdorf I, Brandenburg E, Stark R (2018) Visual feedback for grasping in virtual reality environments for an interface to instruct digital human models. In: Ahram T, Falcão C (eds) Advances in usability and user experience. Springer International Publishing, Cham, pp 228–239
Gentilucci M, Castiello U, Corradini ML et al (1991) Influence of different types of grasping on the transport component of prehension movements. Neuropsychologia 29:361–378. https://doi.org/10.1016/0028-3932(91)90025-4
Harris DJ, Buckingham G, Wilson MR, Vine SJ (2019) Virtually the same? How impaired sensory information in virtual reality may disrupt vision for action. Exp Brain Res 237:2761–2766. https://doi.org/10.1007/s00221-019-05642-8
Holden MK (2005) Virtual environments for motor rehabilitation: review. Cyberpsychol Behav 8:187–211. https://doi.org/10.1089/cpb.2005.8.187
Iyengar V, Santos MJ, Aruin AS (2009) Role of movement velocity on the magnitude of grip force while lifting an object with touch from the contralateral finger. Mot Control 13:130–141. https://doi.org/10.1123/mcj.13.2.130
Jeannerod M (1984) The timing of natural prehension movements. J Mot Behav 16:235–254. https://doi.org/10.1080/00222895.1984.10735319
MacKenzie CL, Graham ED (1997) Separating A and W effects: pointing to targets on computer displays. Behav Brain Sci 20:316–318. https://doi.org/10.1017/S0140525X97361445
MacKenzie CL, Marteniuk RG, Dugas C et al (1987) Three-dimensional movement trajectories in Fitts’ task: implications for control. Q J Exp Psychol Sect A 39:629–647. https://doi.org/10.1080/14640748708401806
Magdalon EC, Michaelsen SM, Quevedo AA, Levin MF (2011) Comparison of grasping movements made by healthy subjects in a 3-dimensional immersive virtual versus physical environment. Acta Psychol (Amst) 138:126–134. https://doi.org/10.1016/j.actpsy.2011.05.015
McIntosh RD, Mon-Williams M, Tresilian JR (2018) Grasping at laws: speed-accuracy trade-offs in manual prehension. J Exp Psychol Hum Percept Perform 44:1022–1037. https://doi.org/10.1037/xhp0000512
Ogawa N, Narumi T, Hirose M (2018) Object size perception in immersive virtual reality: avatar realism affects the way we perceive. In: 2018 IEEE conf virtual real 3D user interfaces, pp 647–648. https://doi.org/10.1109/VR.2018.8446318
Pacchierotti C, Sinclair S, Solazzi M et al (2017) Wearable haptic systems for the fingertip and the hand: taxonomy, review, and perspectives. IEEE Trans Haptics 10:580–600. https://doi.org/10.1109/TOH.2017.2689006
Paulignan Y, Jeannerod M, MacKenzie C, Marteniuk R (1991a) Selective perturbation of visual input during prehension movements. 2. The effects of changing object size. Exp Brain Res 87:407–420. https://doi.org/10.1007/BF00231858
Paulignan Y, MacKenzie C, Marteniuk R, Jeannerod M (1991b) Selective perturbation of visual input during prehension movements. 1. The effects of changing object position. Exp Brain Res 83:502–512. https://doi.org/10.1007/BF00229827
Prachyabrued M, Borst CW (2012) Visual interpenetration tradeoffs in whole-hand virtual grasping. In: 2012 IEEE symp 3D user interfaces, pp 39–42. https://doi.org/10.1109/3DUI.2012.6184182
Prachyabrued M, Borst CW (2014) Visual feedback for virtual grasping. In: 2014 IEEE symp 3D user interfaces, pp 19–26. https://doi.org/10.1109/3DUI.2014.6798835
Rand MK, Smiley-Oyen AL, Shimansky YP et al (2006a) Control of aperture closure during reach-to-grasp movements in Parkinson’s disease. Exp Brain Res 168:131–142. https://doi.org/10.1007/s00221-005-0073-3
Rand MK, Squire LM, Stelmach GE (2006b) Effect of speed manipulation on the control of aperture closure during reach-to-grasp movements. Exp Brain Res 174:74–85. https://doi.org/10.1007/s00221-006-0423-9
Rand MK, Lemay M, Squire LM et al (2007) Role of vision in aperture closure control during reach-to-grasp movements. Exp Brain Res 181:447–460. https://doi.org/10.1007/s00221-007-0945-9
Rand MK, Shimansky YP, Hossain ABMI, Stelmach GE (2008) Quantitative model of transport-aperture coordination during reach-to-grasp movements. Exp Brain Res 188:263–274. https://doi.org/10.1007/s00221-008-1361-5
Rand MK, Shimansky YP, Hossain ABMI, Stelmach GE (2010) Phase dependence of transport–aperture coordination variability reveals control strategy of reach-to-grasp movements. Exp Brain Res 207:49–63. https://doi.org/10.1007/s00221-010-2428-7
Renner RS, Velichkovsky BM, Helmert JR (2013) The perception of egocentric distances in virtual environments-a review. ACM Comput Surv 46:1–40. https://doi.org/10.1145/2543581.2543590
Rizzo AS, Kim GJ (2005) A SWOT analysis of the field of virtual reality rehabilitation and therapy. Presence Teleoperators Virtual Environ 14:119–146. https://doi.org/10.1162/1054746053967094
Rose FD, Brooks BM, Rizzo AA (2005) Virtual reality in brain damage rehabilitation: review. Cyberpsychol Behav 8:241–262. https://doi.org/10.1089/cpb.2005.8.241
Schultheis MT, Rizzo AA (2001) The application of virtual reality technology in rehabilitation. Rehabil Psychol 46:296–311. https://doi.org/10.1037/0090-5550.46.3.296
Sveistrup H (2004) Motor rehabilitation using virtual reality. J Neuroeng Rehabil 1:10. https://doi.org/10.1186/1743-0003-1-10
Viau A, Feldman AG, McFadyen BJ, Levin MF (2004) Reaching in reality and virtual reality: a comparison of movement kinematics in healthy subjects and in adults with hemiparesis. J Neuroeng Rehabil 1:11. https://doi.org/10.1186/1743-0003-1-11
Welford AT (1968) The fundamentals of skill. Methuen, London
Zahariev MA, MacKenzie CL (2007) Grasping at ‘thin air’: multimodal contact cues for reaching and grasping. Exp Brain Res 180:69–84. https://doi.org/10.1007/s00221-006-0845-4
Zahariev MA, Mackenzie CL (2008) Auditory contact cues improve performance when grasping augmented and virtual objects with a tool. Exp Brain Res 186:619–627. https://doi.org/10.1007/s00221-008-1269-0
Acknowledgements
This work was supported by NIH grants #R01NS085122 and #2R01HD058301, and NSF grants #CBET-1804550 and #CMMI-M3X-1935337, to Eugene Tunik. We thank Alex Hunton and Samuel Berin for developing the VR platform.
Author information
Authors and Affiliations
Contributions
MM, MY, MPF, and GT conceived and designed research; MPF performed experiments; MM, MPF, and MPF analyzed data; MM, MY, MPF, and GT interpreted results of experiments; MM prepared figures; MM drafted the manuscript; MM, MY, MPF, and GT edited and revised manuscript; MM, MY, MPF, and GT approved the final version of the manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that no competing interests exist.
Additional information
Communicated by Francesca Frassinetti.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Mangalam, M., Yarossi, M., Furmanek, M.P. et al. Control of aperture closure during reach-to-grasp movements in immersive haptic-free virtual reality. Exp Brain Res 239, 1651–1665 (2021). https://doi.org/10.1007/s00221-021-06079-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00221-021-06079-8