Abstract
The objective of this study was to investigate the influences of latency (i.e., technical system response time), action modality (button press, voice command) and display modality (head-mounted display, monitor) on the sense of agency (SOA). SOA is the experience of controlling one’s own actions and their corresponding effects in the environment. The N = 31 (48% female, with a mean age of 24) participants had to interact repeatedly with three different objects (lamp, tablet and computer) in a virtual environment (presented on a monitor or via a head-mounted display) by using a voice command or pressing a button to turn the objects on. The objects reacted after a specific technical system response delay (150, 450 and 750 ms). Results showed that the SOA was weaker for actions employing voice commands opposed to button presses, except for the explicit SOA in the monitor condition. Higher latencies diminished the explicit, but not the implicit SOA. Neither the explicit nor the implicit SOA was significantly affected by the display modality. The findings in part support the weighting process of different agency cues of the underlying framework, and we propose to extend this model by a sense of presence. Users seem to react as if they have the impression that they are not able to control the technical system properly if they interact through a voice command. Therefore, human–computer interface designers could take account of our findings regarding the modality of an action by providing additional feedback cues to increase the SOA for interactions with voice interfaces.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Aylett MP, Kristensson PO, Whittaker S, Vazquez-Alvarez Y (2014) None of a CHIind: relationship counselling for HCI and speech technology. In: Proceedings of the 2014 ACM annual conference on human factors in computing systems-CHI ’14. https://doi.org/10.1145/2559206.2578868
Berberian B, Sarrazin J-C, Le Blaye P, Haggard P (2012) Automation technology and sense of control: a window on human agency. PLoS ONE 7(3):e34075. https://doi.org/10.1371/journal.pone.0034075
Blakemore S-J, Wolpert DM, Frith CD (2002) Abnormalities in the awareness of action. Trends Cogn Sci 6(6):237–242. https://doi.org/10.1016/s1364-6613(02)01907-1
Blom KJ, Beckhaus S (2014) The design space of dynamic interactive virtual environments. Virtual Real 18(2):101–116. https://doi.org/10.1007/s10055-013-0232-y
Borsci S, Lawson G, Salanitri D, Jha B (2016) When simulated environments make the difference: the effectiveness of different types of training of car service procedures. Virtual Real 20(2):83–99. https://doi.org/10.1007/s10055-016-0286-8
Cohen J (1992) A power primer. Psychol Bull 112(1):155–159. https://doi.org/10.1037/0033-2909.112.1.155
Coyle D, Moore J, Kristensson PO, Fletcher P, Blackwell A (2012) I did that!. In: Proceedings of the 2012 ACM annual conference on human factors in computing systems-CHI ’12. https://doi.org/10.1145/2207676.2208350
Ebert JP, Wegner DM (2010) Time warp: authorship shapes the perceived timing of actions and events. Conscious Cognit 19(1):481–489. https://doi.org/10.1016/j.concog.2009.10.002
Engbert K, Wohlschläger A, Thomas R, Haggard P (2007) Agency, subjective time, and other minds. J Exp Psychol Hum Percept Peform 33(6):1261–1268. https://doi.org/10.1037/0096-1523.33.6.1261
Engbert K, Wohlschläger A, Haggard P (2008) Who is causing what? The sense of agency is relational and efferent-triggered. Cognition 107(2):693–704. https://doi.org/10.1016/j.cognition.2007.07.021
Evans N, Gale S, Schurger A, Blanke O (2015) Visual feedback dominates the sense of agency for brain-machine actions. PLoS ONE 10(6):e0130019. https://doi.org/10.1371/journal.pone.0130019
Farrer C, Valentin G, Hupé JM (2013) The time windows of the sense of agency. Conscious Cognit 22(4):1431–1441. https://doi.org/10.1016/j.concog.2013.09.010
Field AP (2009) Discovering statistics using IBM SPPS statistics and sex and drugs and rock “n” roll, 3rd edn. Sage Publications, Thousand Oaks
Haggard P, Clark S, Kalogeras J (2002) Voluntary action and conscious awareness. Nat Neurosci 5(4):382–385. https://doi.org/10.1038/nn827
Havranek M, Langer N, Cheetham M, Jäncke L (2012) Perspective and agency during video gaming influences spatial presence experience and brain activation patterns. Behav Brain Funct 8(1):34. https://doi.org/10.1186/1744-9081-8-34
Ijsselsteijn WA, de Kort YAW, Haans A (2006) Is this my hand I see before me? The rubber hand illusion in reality, virtual reality, and mixed reality. Presence Teleoper Virtual Environ 15(4):455–464. https://doi.org/10.1162/pres.15.4.455
Kawabe T (2013) Inferring sense of agency from the quantitative aspect of action outcome. Conscious Cognit 22(2):407–412. https://doi.org/10.1016/j.concog.2013.01.006
Kawabe T, Roseboom W, Nishida S (2013) The sense of agency is action-effect causality perception based on cross-modal grouping. Proc R Soc B Biol Sci 280:20130991. https://doi.org/10.1098/rspb.2013.0991
Kilteni K, Groten R, Slater M (2012) The sense of embodiment in virtual reality. Presence Teleoper Virtual Environ 21(4):373–387. https://doi.org/10.1162/pres_a_00124
Lee M, Billinghurst M, Baek W, Green R, Woo W (2013) A usability study of multimodal input in an augmented reality environment. Virtual Real 17(4):293–305. https://doi.org/10.1007/s10055-013-0230-0
Limerick H, Moore JW, Coyle D (2015) Empirical evidence for a diminished sense of agency in speech interfaces. In: Proceedings of the 33rd annual ACM conference on human factors in computing systems-CHI ’15. https://doi.org/10.1145/2702123.2702379
Moore JW, Obhi SS (2012) Intentional binding and the sense of agency: a review. Conscious Cognit 21(1):546–561. https://doi.org/10.1016/j.concog.2011.12.002
Moore JW, Middleton D, Haggard P, Fletcher PC (2012) Exploring implicit and explicit aspects of sense of agency. Conscious Cognit 21(4):1748–1753. https://doi.org/10.1016/j.concog.2012.10.005
Obhi SS, Hall P (2011) Sense of agency in join action: influence of human and computer co-actors. Exp Brain Res 211:663–670. https://doi.org/10.1007/s00221-011-2662-7
Schoeffler M, Gernert JL, Neumayer M, Westphal S, Herre J (2015) On the validity of virtual reality-based auditory experiments: a case study about ratings of the overall listening experience. Virtual Real 19(3):181–200. https://doi.org/10.1007/s10055-015-0270-8
Shneiderman B, Plaisant C (2004) Designing the user interface: strategies for effective human–computer interaction, 4th edn. Pearson Education, London
Slater M, Lotto B, Arnold MM, Sánchez-Vives MV (2009) How we experience immersive virtual environments: the concept of presence and its measurement. Anu Psicol 40(2):193–210
Synofzik M, Thier P, Lindner A (2006) Internalizing agency of self-action: perception of one’s own hand movements depends on an adaptable prediction about the sensory action outcome. J Neurophysiol 96(3):1592–1601. https://doi.org/10.1152/jn.00104.2006
Synofzik M, Vosgerau G, Newen A (2008a) I move, therefore I am: a new theoretical framework to investigate agency and ownership. Conscious Cognit 17(2):411–424. https://doi.org/10.1016/j.concog.2008.03.008
Synofzik M, Vosgerau G, Newen A (2008b) Beyond the comparator model: a multifactorial two-step account of agency. Conscious Cognit 17(1):219–239. https://doi.org/10.1016/j.concog.2007.03.010
Synofzik M, Vosgerau G, Lindner A (2009) Me or not me—an optimal integration of agency cues? Conscious Cognit 18(4):1065–1068. https://doi.org/10.1016/j.concog.2009.07.007
Synofzik M, Vosgerau G, Voss M (2013) The experience of agency: an interplay between prediction and postdiction. Front Psychol. https://doi.org/10.3389/fpsyg.2013.00127
Turchet L (2015) Designing presence for real locomotion in immersive virtual environments: an affordance-based experiential approach. Virtual Real 19(3):277–290. https://doi.org/10.1007/s10055-015-0267-3
Vorderer P, Wirth W, Gouveia FR, Biocca F, Saari T, Jäncke F, Böcking S, Schramm H, Gysbers A, Hartmann T, Klimmt C, Laarni J, Ravaja N, Sacau A, Baumgartner T, Jäncke P (2004) MEC spatial presence questionnaire (MEC-SPQ): short documentation and instructions for application. Report to the European Community, project presence: MEC (IST-2001-37661). https://academic.csuohio.edu/kneuendorf/frames/MECFull.pdf. Accessed 10 Sept 2019
Wirth W, Hartmann T, Böcking S, Vorderer P, Klimmt C, Schramm H, Saari T, Laarni J, Ravaja N, Gouveia FR, Biocca F, Sacau A, Jäncke L, Baumgartner T, Jäncke P (2007) A process model of the formation of spatial presence experiences. Media Psychol 9(3):493–525. https://doi.org/10.1080/15213260701283079
Author information
Authors and Affiliations
Contributions
PW and PS designed the study, conducted the experiment and wrote as well as edited the manuscript in equal parts. NR and TF supported in study design and supported and reviewed manuscript preparation. JK supported the general research project and reviewed the manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Winkler, P., Stiens, P., Rauh, N. et al. How latency, action modality and display modality influence the sense of agency: a virtual reality study. Virtual Reality 24, 411–422 (2020). https://doi.org/10.1007/s10055-019-00403-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10055-019-00403-y