Collaborating with my Doppelgänger: The Effects of Self-similar Appearance and Voice of a Virtual Character during a Jigsaw Puzzle Co-solving Task

The research community has long been interested in human interaction with embodied virtual characters in virtual reality (VR). At the same time, interaction with self-similar virtual characters, or virtual doppelgängers, has become a prominent topic in both VR and psychology due to the intriguing psychological effects these characters can have on people. However, studies on human interaction with self-similar virtual characters are still limited. To address this research gap, we designed and conducted a 2 (appearance: self-similar vs. non-self-similar appearance) × 2 (voice: self-similar vs. non-self-similar voice) within-group study (N = 25) to explore how combinations of appearance and voice factors influence participants' perception of virtual characters. During the study, we asked participants to collaborate with a virtual character in solving a VR jigsaw puzzle. After each experimental condition, we had participants complete a survey about their experiences with the virtual character. Our findings showed that 1) the virtual characters' self-similarity in appearance enhanced the sense of co-presence and perceived intelligence, but it also elicited higher eeriness; 2) the self-similar voices led to higher ratings on the characters' likability and believability; however, they also induced a more eerie sensation; and 3) we observed an interaction effect between appearance and voice factors for ratings on believability, where the virtual characters were considered more believable when their self-similarity in appearance matched that of their voices. This study provided valuable insights and comprehensive guidance for creating novel collaborative experiences with self-similar virtual characters in immersive environments.