research-article

Blowing in the wind: : Increasing social presence with a virtual human via environmental airflow interaction in mixed reality

Authors:

Jason Hochreiter,

Gregory WelchAuthors Info & Claims

Volume 83, Issue C

Pages 23 - 32

https://doi.org/10.1016/j.cag.2019.06.006

Published: 01 October 2019 Publication History

Highlights

•

Importance of social presence in real-virtual human interactions.

•

Physical-virtual interaction to improve social presence in mixed reality.

•

Improving social presence via plausible physical-virtual airflow interaction.

•

Improving social presence via appropriate virtual human environment-aware behavior.

Graphical abstract

Display Omitted

Abstract

In this paper, we describe two human-subject studies in which we explored and investigated the effects of subtle multimodal interaction on social presence with a virtual human (VH) in mixed reality (MR). In the studies, participants interacted with a VH, which was co-located with them across a table, with two different platforms: a projection based MR environment and an optical see-through head-mounted display (OST-HMD) based MR environment. While the two studies were not intended to be directly comparable, the second study with an OST-HMD was carefully designed based on the insights and lessons learned from the first projection-based study. For both studies, we compared two levels of gradually increased multimodal interaction: (i) virtual objects being affected by real airflow (e.g., as commonly experienced with fans during warm weather), and (ii) a VH showing awareness of this airflow. We hypothesized that our two levels of treatment would increase the sense of being together with the VH gradually, i.e., participants would report higher social presence with airflow influence than without it, and the social presence would be even higher when the VH showed awareness of the airflow. We observed an increased social presence in the second study when both physical–virtual interaction via airflow and VH awareness behaviors were present, but we observed no clear difference in participant-reported social presence with the VH in the first study. As the considered environmental factors are incidental to the direct interaction with the real human, i.e., they are not significant or necessary for the interaction task, they can provide a reasonably generalizable approach to increase social presence in HMD-based MR environments beyond the specific scenario and environment described here.

References

[1]

J. Blascovich, Social influence within immersive virtual environments, in: Schroeder R. (Ed.), The social life of avatars, Springer London, 2002, pp. 127–145.

[2]

M. Garau, M. Slater, V. Vinayagamoorthy, A. Brogni, A. Steed, M.A. Sasse, The impact of avatar realism and eye gaze control on perceived quality of communication in a shared immersive virtual environment, Proceedings of the ACM SIGCHI conference on human factors in computing systems, 2003, pp. 529–536.

[3]

L.P. Morency, I. de Kok, J. Gratch, A probabilistic multimodal approach for predicting listener backchannels, Auton Agents Multi-Agent Syst 20 (1) (2009) 70–84.

[4]

J. Allwood, An activity based approach to pragmatics, in: Bunt H., Black W. (Eds.), Abduction, belief and context in dialogue: studies in computational pragmatics, John Benjamins Publishing Company, 2000, pp. 47–80.

[5]

Microsoft. Spatial mapping (windows mixed reality). https://developer.microsoft.com/en-us/windows/mixed-reality/spatial_mapping; 2017.

[6]

K. Kim, D. Maloney, G. Bruder, J.N. Bailenson, G.F. Welch, The effects of virtual human’s spatial and behavioral coherence with physical objects on social presence in AR, Comput Animat Virtual Worlds 28 (3–4) (2017) e1771.

[7]

M. Lee, K. Kim, S. Daher, A. Raij, R. Schubert, J. Bailenson, et al., The wobbly table: increased social presence via subtle incidental movement of a real-virtual table, Proceedings of the IEEE virtual reality, 2016, pp. 11–17.

[8]

K. Kim, R. Schubert, G. Welch, Exploring the impact of environmental effects on social presence with a virtual human, Proceedings of international conference on intelligent virtual agents, 10011, 2016, pp. 470–474.

[9]

K. Kim, G. Bruder, G.F. Welch, Blowing in the wind: increasing copresence with a virtual human via airflow influence in augmented reality, Proceedings of the international conference on artificial reality and telexistence Eurographics symposium on virtual environments, 2018, pp. 183–190.

[10]

J.N. Bailenson, A.C. Beall, J. Loomis, J. Blascovich, M. Turk, Transformed social interaction, augmented gaze, and social influence in immersive virtual environments, Hum Commun Res 31 (4) (2005) 511–537.

[11]

J.H. Chuah, A. Robb, C. White, A. Wendling, S. Lampotang, R. Kopper, et al., Exploring agent physicality and social presence for medical team training, Presence Teleoperators Virtual Environ 22 (2) (2013) 141–170.

[12]

L. Huang, L.-P. Morency, J. Gratch, Virtual rapport 2.0, Proceedings of the international conference on intelligent virtual agents, 6895, 2011, pp. 68–79.

[13]

C.S. Oh, J.N. Bailenson, G.F. Welch, A systematic review of social presence: definition, antecedents, and implications, Front Robot AI 5 (114) (2018) 1–35.

[14]

S. Zhao, Toward a taxonomy of copresence, Presence Teleoperators Virtual Environ 12 (5) (2003) 445–455.

[15]

M. Slater, Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments, Philos Trans R Soc Lond Ser B Biol Sci 364 (1535) (2009) 3549–3557.

[16]

C. Harms, F. Biocca, Internal consistency and reliability of the networked minds measure of social presence, Proceedings of the seventh annual international presence workshop, 2004, pp. 246–251.

[17]

J. Blascovich, J. Loomis, A.C. Beall, K.R. Swinth, C.L. Hoyt, J.N. Bailenson, Immersive virtual environment technology as a methodological tool for social psychology, Psychol Inquiry 13 (2) (2002) 103–124.

[18]

E. Goffman, Behavior in public places: notes on the social organization of gatherings, The Free Press, New York, 1963.

[19]

H.Q. Dinh, N. Walker, C. Song, A. Kobayashi, L.F. Hodges, Evaluating the importance of multi-sensory input on memory and the sense of presence in virtual environments, Proceedings of the IEEE virtual reality, 1999, pp. 222–228.

[20]

T. Moon, G.J. Kim, Design and evaluation of a wind display for virtual reality, Proceedings of the ACM symposium on virtual reality software and technology, 2004, pp. 122–128.

[21]

F. Hülsmann, J. Fröhlich, N. Mattar, I. Wachsmuth, Wind and warmth in virtual reality: implementation and evaluation, Proceedings of the virtual reality international conference, 2014, pp. 28:1–8.

[22]

M. Feng, A. Dey, R.W. Lindeman, The effect of multi-sensory cues on performance and experience during walking in immersive virtual environments, Proceedings of the IEEE virtual reality, 2016, pp. 173–174.

[23]

A. Lehmann, C. Geiger, B. Wöldecke, J. Stöcklein, Poster: design and evaluation of 3d content with wind output, Proceedings of the IEEE symposium on 3D user interfaces, 2009, pp. 151–152.

[24]

L. Deligiannidis, R.J. Jacob, The VR scooter: Wind and tactile feedback improve user performance, Proceedings of the IEEE symposium on 3d user interfaces, 2006, pp. 143–150.

[25]

L.A. Dieker, J.A. Rodriguez, B. Lignugaris/Kraft, M.C. Hynes, C.E. Hughes, The potential of simulated environments in teacher education: current and future possibilities, Teach Educ Spec Educ J Teach Educ Div Counc Except Child 37 (1) (2013) 21–33.

[26]

A. Rizzo, S. Scherer, D. Devault, J. Gratch, R. Artstein, A. Hartholt, et al., Detection and computational analysis of psychological signals using a virtual human interviewing agent, Int J Disabil Hum Dev 15 (3) (2014).

[27]

M. Hoque, M. Courgeon, J.-C. Martin, B. Mutlu, R.W. Picard, Mach: my automated conversation coach, Proceedings of the ACM international joint conference on pervasive and ubiquitous computing, 2013, pp. 697–706.

[28]

J. Hochreiter, S. Daher, A. Nagendran, L. Gonzalez, G. Welch, Optical touch sensing on nonparametric rear-projection surfaces for interactive physical-virtual experiences, Presence Teleoperators Virtual Environ 25 (1) (2016) 33–46.

[29]

S. Andrist, M. Gleicher, B. Mutlu, Looking coordinated: bidirectional gaze mechanisms for collaborative interaction with virtual characters, Proceedings of the ACM CHI conference on human factors in computing systems, 2017, pp. 2571–2582.

[30]

K. Kim, A. Nagendran, J. Bailenson, G. Welch, Expectancy violations related to a virtual human’s joint gaze behavior in real-virtual human interactions, Proceedings of the international conference on computer animation and social agents, 2015, pp. 5–8.

[31]

K. Kim, L. Boelling, S. Haesler, J.N. Bailenson, G. Bruder, G.F. Welch, Does a digital assistant need a body? The influence of visual embodiment and social behavior on the perception of intelligent virtual agents in AR, Proceedings of IEEE international symposium on mixed and augmented reality, 2018, pp. 105–114.

[32]

J.N. Bailenson, J. Blascovich, A.C. Beall, J.M. Loomis, Interpersonal distance in immersive virtual environments, Personal Soc Psychol Bull 29 (7) (2003) 819–833.

[33]

A. Frischen, A.P. Bayliss, S.P. Tipper, Gaze cueing of attention: visual attention, social cognition, and individual differences, Psychol Bull 133 (4) (2007) 694–724.

[34]

C. Basdogan, C.-H. Ho, M.A. Srinivasan, M. Slater, An experimental study on the role of touch in shared virtual environments, ACM Trans Comput Hum Interact 7 (4) (2000) 443–460.

Digital Library

[35]

M. Lee, G. Bruder, T. Höllerer, G. Welch, Effects of unaugmented periphery and vibrotactile feedback on proxemics with virtual humans in AR, IEEE Trans Vis Comput Gr 24 (4) (2018) 1525–1534.

[36]

K. Kim, M. Billinghurst, G. Bruder, H. Been-Lirn Duh, G.F. Welch, Revisiting trends in augmented reality research: a review of the 2nd decade of ISMAR (2008–2017), IEEE Trans Vis Comput Gr 24 (11) (2018) 2947–2962.

[37]

N. Norouzi, K. Kim, J. Hochreiter, M. Lee, S. Daher, G. Bruder, et al., A systematic survey of 15 years of user studies published in the intelligent virtual agents conference, Proceedings of the ACM international conference on intelligent virtual agents, 2018, pp. 17–22.

[38]

A. Dey, M. Billinghurst, R.W. Lindeman, J.E. Swan, A systematic review of 10 years of augmented reality usability studies: 2005 to 2014, Front Robot AI 5 (37) (2018) 1–28.

[39]

M. Lee, N. Norouzi, G. Bruder, P.J. Wisniewski, G.F. Welch, The physical-virtual table: exploring the effects of a virtual human’s physical influence on social interaction, Proceedings of the 24th ACM symposium on virtual reality software and technology, 2018, pp. 1–11.

[40]

N. Norouzi, G. Bruder, B. Belna, S. Mutter, D. Turgut, G. Welch, A systematic review of the convergence of augmented reality, intelligent virtual agents, and the internet of things, Springer transactions on computational science computational intelligence, 2019, pp. 1–24.

Cited By

Chang ZPai YCao JGupta KBillinghurst M(2024)EMiRAs-Empathic Mixed Reality AgentsProceedings of the 3rd Empathy-Centric Design Workshop: Scrutinizing Empathy Beyond the Individual10.1145/3661790.3661791(1-7)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3661790.3661791
Norouzi NKim KBruder GBailenson JWisniewski PWelch G(2022)The advantages of virtual dogs over virtual peopleInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2022.102838165:COnline publication date: 1-Sep-2022
https://dl.acm.org/doi/10.1016/j.ijhcs.2022.102838
Caldas OSanchez NMauledoux MAvilés ORodriguez-Guerrero C(2022)Leading presence-based strategies to manipulate user experience in virtual reality environmentsVirtual Reality10.1007/s10055-022-00645-326:4(1507-1518)Online publication date: 1-Dec-2022
https://dl.acm.org/doi/10.1007/s10055-022-00645-3

Index Terms

Blowing in the wind: Increasing social presence with a virtual human via environmental airflow interaction in mixed reality
1. Computing methodologies
  1. Computer graphics
    1. Graphics systems and interfaces
      1. Virtual reality
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Mixed / augmented reality
      2. Virtual reality

Index terms have been assigned to the content through auto-classification.

Recommendations

An evaluation of spatial presence, social presence, and interactions with various 3D displays
CASA '16: Proceedings of the 29th International Conference on Computer Animation and Social Agents

This paper presents an immersive volleyball game, where a player plays not only against virtual opponents but also with support on his/her side of virtual teammates. This volleyball game has been implemented for several 3D displays such as a ...
Improving Social Presence with a Virtual Human via Multimodal Physical -- Virtual Interactivity in AR
CHI EA '18: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems

In a social context where a real human interacts with a virtual human (VH) in the same space, one's sense of social/co-presence with the VH is an important factor for the quality of interaction and the VH's social influence to the human user in context. ...
The effect of affective iconic realism on anonymous interactants' self-disclosure
CHI EA '09: CHI '09 Extended Abstracts on Human Factors in Computing Systems

In this paper, we describe progress in research designed to explore the effect of the combination of avatars' visual fidelity and users' anticipated future interaction on self-disclosure in emotionally engaged and synchronous communication. We ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Computers and Graphics

Computers and Graphics Volume 83, Issue C

Oct 2019

122 pages

ISSN:0097-8493

Issue’s Table of Contents

Elsevier Ltd.

Publisher

Pergamon Press, Inc.

United States

Publication History

Published: 01 October 2019

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Chang ZPai YCao JGupta KBillinghurst M(2024)EMiRAs-Empathic Mixed Reality AgentsProceedings of the 3rd Empathy-Centric Design Workshop: Scrutinizing Empathy Beyond the Individual10.1145/3661790.3661791(1-7)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3661790.3661791
Norouzi NKim KBruder GBailenson JWisniewski PWelch G(2022)The advantages of virtual dogs over virtual peopleInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2022.102838165:COnline publication date: 1-Sep-2022
https://dl.acm.org/doi/10.1016/j.ijhcs.2022.102838
Caldas OSanchez NMauledoux MAvilés ORodriguez-Guerrero C(2022)Leading presence-based strategies to manipulate user experience in virtual reality environmentsVirtual Reality10.1007/s10055-022-00645-326:4(1507-1518)Online publication date: 1-Dec-2022
https://dl.acm.org/doi/10.1007/s10055-022-00645-3

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents