Abstract

Introduction

Face-to-face (FF) interaction is one of the most common types of social interaction in our everyday life. With crucial external information (e.g. facial expression or body gestures) and rapid real-time feedback, it reduces the uncertainty about other people’s intention. Thus, FF interaction facilitates shared intentionality, in which people share psychological (cognitive, affective and motivational) states with each other (Gilbert, 1989; Searle, 1995; Tuomela, 1995), leading to better understanding and collaborative social interactions (Bordia, 1997; Rocco, 1998; Valley et al., 1998; Hill et al., 2009).

Establishing shared intentionality during social interactions requires mentalizing (Frith and Frith, 2001), an ability that enables people to infer other’s intention, beliefs and goals (Hari and Kujala, 2009). Mentalizing recruits a sophisticated neural network, including medial prefrontal cortex, posterior cingulate and temporo-parietal junction (TPJ) (Frith and Frith, 2006; Frith, 2007). Within the mentalizing network, the right TPJ (rTPJ) takes a critical role in establishing social context for behavior (Carter and Huettel, 2013) by enabling inferences about other agents’ intentions (Saxe, 2006). For example, the rTPJ is activated in reading character’s thoughts in stories (Saxe and Powell, 2006), engages differently in strategic deception in a two-person bargaining game (Bhatt et al., 2010), and mediates joint attention between a pair of people during FF interaction (Redcay et al., 2010). Disrupting the rTPJ with transcranial magnetic stimulation causes deficits in inferring intentions and beliefs (Young et al., 2010), while activating rTPJ using transcranial direct current stimulation enhances the on-line control of self-other representations (Santiesteban et al., 2012).

How to investigate the neural signatures of two-person social interactions has been on the forefront of social neuroscience during the last decade. One big challenge is to capture the naturally dynamic process of real interaction; another is to measure the neural signals of two persons’ brain interacting simultaneously (Hari and Kujala, 2009). Previous research has combined two-person interactive economic exchange paradigms with functional MRI (fMRI) (Montague et al., 2002; King-Casas et al., 2005; Krueger et al., 2007), electroencephalography (EEG) (Babiloni et al., 2007a,b; Yun et al., 2008; Astolfi et al., 2010a,b; Fallani et al., 2010) and functional near-infrared spectroscopy (fNIRS) (Cui et al., 2012; Jiang et al., 2012) hyperscanning to solve these two problems. During naturally occuring FF interaction, neural synchronization between couple of players has been found in two-person games, including simple cooperation and competition task (Funane et al., 2011; Cui et al., 2012), prisoner’s dilemma game, card game and ultimatum game (Babiloni et al., 2007a; Yun et al., 2008; Astolfi et al., 2010a). Since mentalizing has crucial function in interaction, exploring the neural signatures of mentalizing system in FF games with hyperscanning method could provide more information about the process of social interaction.

In this study, we investigated how FF interaction impacts interpersonal brain synchronization during economic exchange. We combined a revised ultimatum game with fNIRS hyperscanning, in which pairs of strangers either interacted FF or face-blocked (FB) while their interpersonal brain activations were recorded in the rTPJ (target region) and right dorsolateral PFC (rDLPFC) (control region), a region crucially involved in strategic decision-making during two-person economic exchange (Sanfey et al., 2003; Knoch et al., 2009; Bhatt et al., 2010). We hypothesized that FF compared to FB interaction leads to greater shared intentionality between pairs of partners due to access of crucial external information (e.g. facial expression, body gestures) and results into greater cooperation. We further predicted that greater shared intentionality between partners is mirrored by greater interpersonal brain synchronizations in the rTPJ during FF compared to FB interaction due to its role in inferring others’ intentions, beliefs and goals.

Materials and methods

Procedure

Pairs of strangers sat across two tables carrying two computer displays and interacted repeatedly in an adapted version of the ultimatum game (UG) (Güth et al., 1982). In addition to the classic UG (a proposer (P) makes an offer and a responder (R) can either accept or reject the offer), we added two new stages to the game: (i) a stage where P had a chance to deceive R before making the offer and (ii) a stage where P had to judge whether R would accept or reject the offer and R had to judge whether P had told the truth or a lie. Adding these two stages provided a chance to measure how successful pairs of partners understood each other’s intention during economic exchange. Pairs were randomly assigned into two condition: (i) in the FF condition pairs needed to look at each other’s face and P communicated the total amount and the offer orally to make interaction more ecologically valid (Note that we asked P to communicate only information regarding the total amount and the offer: for example, P told R that ‘I got 10’ and ‘I give you 5’; and R did not speak at all during the experiment but responded through button presses instead.); (ii) in the FB condition they were separated by a board and communicated through binary decisions through button presses on their computers (Figure 1A). All manipulations of button presses were same under two conditions.

Open in a separate window

Fig. 1.

Experimental design. (A) Type of interaction. Pairs of players (proposer [P], responder [R]) interacted in either face-to-face (FF) or FB (blocked with a board, FB). (B) Experimental procedure. The game was composed of six stages: after P sees the true total amount (stage 1), P tells R how much the total amount is (stage 2), P makes an offer to R (stage 3), both P and R judge each other’s actions (stage 4), R decides to accept or reject (stage 5), both P and R see the outcome (stage 6). The analyses were focused on stage 2 to 4 (red dotted line frame) during the mutual interaction between P and R. (C) fNIRS set up. NIRS data were recorded from the right TPJ and DLPFC localized by the international 10-20 system. Red dots are the emitters and blue dots are the detectors. Measured channels (CH) between pairs of emitter and detector were marked as red numbers. rTPJ was marked (yellow) with the MNI coordinate from previous studies (Saxe and Powell, 2006; Redcay et al., 2010).

At the beginning of the experiment, players were randomly assigned to fixed roles, either as P or R. Then a participant assigned as P was paired with a participant assigned as R and they played together for 54 rounds separated into three blocks of 18 trials, each lasting about 8minutes. An experimental trial consisted of six stages (Figure 1B). In stage 1 (2s), P received one of six total amounts as an endowment (Monetary units [MUs]: 4, 6, 8, 10, 12, or 14). In stage 2 (5s), P communicated the total amount to R with a chance to deceive (Honesty Rate, HR) (e.g. P received 14 MUs, but told R the total amount was 10 MUs). In stage 3 (5s), P split the total amount between players (Offer Proportion, OP (percentage of offer to true total amount); Fairness Rate, FR (percentage of trials that perceived as fair by R if is exactly 50% of the communicated total amount) (e.g. P gave R 5 MUs and kept 5 MUs). In stage 4 (5s), both players judged each other’s actions (with binary button presses: yes or no): P judged whether R would accept the offer or not and R judged whether P had told the truth or not. To determine players’ intention to each others’ behavior and its relationship to cooperation, we calculated the percentage of trials that both players positively judged each other’s action (i.e. P judged that R would accept the offer and R judged P did not tell a lie) for a total of 54 trials as the shared intentionality rate (SIR). In stage 5 (5s), R decided whether to accept or reject the offer by a binary button press (yes, no) (rejection rate, RR). If R accepted the offer, both players received the allocated total amount; otherwise, both received nothing. In stage 6 (5s), the final (‘true’) results were revealed to both players on the screen (e.g. P actually received 9 MUs, instead of 5 MU, and R 5 MUs). For the FF condition, FF interaction was only required through stages 2 to 4 (i.e. in other stages, either P or R needed to check the screen for information). Further, participants were asked to complete the Positive and Negative Affect Schedule (PANAS) scale (Watson et al., 1988) to measure emotional effects during the dynamic interactions in this game, which includes positive items (e.g. attentive, interested, alert, excited, enthusiastic, inspired, proud, determined, strong and active) and negative items (e.g. distressed, upset, hostile, irritable, scared, afraid, ashamed, guilty and nervous, jittery). They also filled the interpersonal reactivity index (IRI) scale (Davis, 1980) to control individuals’ ability to adopt the partners’ perspectives and feelings in this game. In this scale, perspective taking (PT) measures the tendency to spontaneously adopt other people’s perspectives and see things on their sides; fantasy (F) assesses predisposition to image the feelings and actions of characters in books, movies and plays into themselves; empathic concern (EC) assesses the tendency to experience feelings of sympathy and compassion for unfortunate others; personal distress (PD) assesses the ‘self-oriented’ feelings of personal anxiety and discomfort in interpersonal settings. Finally, participants were told that their earned MUs would be transferred into real money based on their performance during game to incentivize real behaviors before the experiment. After participants finished the experiment and questionnaires, we debriefed and paid them privately around 25 RMB (about $4) depending on their performance and additional compensation for their participation.

Data analysis

Behavioral data analysis was carried out using SPSS 17.0 (IBM, New York, NY, USA) with alpha set to P<0.05 (two-tailed). Behavioral data were normally distributed (Kolmogorov–Smirnov test) and homogeneous in variance (Bartlett’s test). Mixed 3×2 analyses of variance (ANOVAs) on behavioral game measures (HR, OP, FR, SIR, RR) were performed with Time (Block1, Block2, Block3) as a within-subjects factor and Condition (FF, FB) as a between-subjects factor (Bonferroni corrected). Independent-samples t-tests for gains and empathy measures were performed between FF and FB (Figure 2). Effect sizes for ANOVAs [partial Eta squared; (η_p²) with 0.01, 0.06, and 0.14 as small-, medium- and large-sized effects] and Cohen’s d for t-tests (with 0.2, 0.5 and 0.8 as small-, medium- and large-sized effects) were calculated.

Open in a separate window

Fig. 2.

Behavioral Results. (A) Game measures. The comparison between face-to-face (FF) and face-block (FB) interactions only revealed significant differences for shared intentionality rate and rejection rate, but not for honest rate and fairness rate. (B) RT (response time) of and Gains in Game. Pairs made positive judgment of each other’s behavior (shared intentionality) more quickly and gained more money in FF than in FB. (C) Time effect. Shared intentionality rate decreased across blocks in FB but not in FF (*P<0.05, **P<0.01). Error bars indicate standard errors.

For the individuals’ fNIRS data analysis, after preprocessed with initial time baseline correction, hrf low-pass filtering and wavelet minimum description length detrending in NIRS-SPM (Jang et al., 2009; Ye et al., 2009; Tak et al., 2011), time series of oxy-Hb concentration changes of each channel of each subject were normalized into z-scores with means and standardizations. We tested oxy-Hb changes’ tendency of differences between conditions on stages 2–4 across all channels with ANOVA analysis and bivariate Pearson correlations between proposers’ and responders’ oxy-Hb concentration changes and their behaviors (HR, OP, FR, RR).

Furthermore, we focused on the analysis of synchronization between two players’ fNIRS data. A 3-min resting-state session was performed as a baseline before the experimental session, during which participants were instructed to relax and remain motionless as much as possible with eyes closed sitting in a silent environment (Lu et al., 2010; Jiang et al., 2012). During the experiment, the first 30 s of data were discarded for each block, resulting in ~500 s of task data per block. fNIRS oxy-Hb time series on stages 2–4 (stages require FF interactions in FF condition) were used to calculate the interpersonal brain coherence as synchronization between pairs. We applied the Wavelet Transform Coherence (WTC) method (Torrence and Compo, 1998; Grinsted et al., 2004; Murphya et al., 2009) and utilized the wavelet coherence MATLAB© package (http://noc.ac.uk/using-science/crosswavelet-wavelet-coherence) (Grinsted et al., 2004). The wavelet coherence measures the correlation between two signals’ component on both frequency and time, and it has been successfully applied in two previous fNIRS hyperscanning studies (Cui et al., 2012; Jiang et al., 2012). An example of raw data and WTC of one pair in our study were shown in Figure 3A and B. False discovery rate (FDR) correction was used to minimize the multiple comparison problem.

Open in a separate window

Fig. 3.

fNIRS results. (A) Raw oxy-Hb time courses. Raw oxy-Hb time courses of Proposer (P) (blue) and Responder (R) (red) in first ten trials in one block from channels in rTPJ in FF (i.e. raw oxy-Hb time courses in Channel 14 of P and R in a representative pair). (B) Synchronization indicated by coherence. Wavelet transform coherence (WTC) based on raw oxy-Hb signal from channel 14 from P and R in the same representative pair in FF. The higher coherence encoded by red is in the task frequency band (12.8–51.2s). (C) One-sample t-test of interpersonal brain synchronization. CH5 and CH14 in the FF, CH19 in the FB condition showed significant synchronization (FDR corrected). (D) Comparison of synchronization between conditions. Only CH14 but not CH5, CH19 or any channel in rDLPFC (bars for CH16, CH8, and CH13 are given as samples) demonstrated notably greater synchronization in FF compared to FB (*P<0.05). Error bars indicate standard errors.

Based on the WTC analysis of two time series (Figure 3A), we found that the frequency band between 12.8 and 51.2s (between 0.02 and 0.08Hz) was more sensitive to our task and could remove high and low frequency noise as well. Then average coherence value in this band in the experimental session was computed to subtract the average coherence in the initial resting-state session (Cui et al., 2012; Jiang et al., 2012). After the coherence values were converted into a z-scores (Fisher z-transformation) (Chang and Glover, 2010), we performed one-sampled t-tests (P<0.05, two-tailed, FDR corrected) for each channel for FF and FB conditions separately to identify significantly synchronized channels during the experiment. For channels with significant synchronization, independent-samples t-tests were computed to identify significant difference between FF and FB conditions. Next, for those significant channels between FF and FB condition, mixed 3×2 ANOVAs on coherence measures were performed with Time (Block1, Block2, Block3) as a within-subjects factor and Condition (FF, FB) as a between-subjects factor. Finally, bivariate Pearson correlations between coherence and shared intentionality measures were computed to reveal brain-behavior associations.

Results

FNIRS results

Overall, z-scores of players’ oxy-Hb concentration changes showed a significantly decreasing tendency in FF than in FB across all channels (F(1,95)=11.44, P=0.001, η_p²=0.11). For the correlation between oxy-Hb changes and behaviors, some channels in rTPJ region showed significant correlations only in FF, particularly for the responder. That is, the oxy-Hb of responder in CH14 and CH19 were negatively correlated with HR and OP, and positively correlated to RR (ps<0.04) in FF but not in FB (ps>0.09) (Table 1).

Table 1.

Correlation between proposers’ and responders’ oxy-Hb concentration changes (z-scores) and their behaviors in rTPJ region

Channel	Correlation(r) between Oxy-Hb changes and behaviors
	HR		OP		FR		RR
	FF	FB	FF	FB	FF	FB	FF	FB
Proposer
CH14	−0.16	−0.25	−0.11	−0.004	−0.35*	−0.25	0.03	0.19
CH19	0.17	−0.22	−0.03	0.08	0.004	−0.20	0.002	0.14
Responder
CH14	−0.35*	−0.03	−0.39**	0.003	−0.05	0.25	0.35*	0.04
CH19	−0.28*	−0.09	−0.30*	0.10	−0.02	0.02	0.30*	0.04

Open in a separate window

FR, fairness rate; HR, honesty rate; OP, offer proportion; RR, rejection rate.

*P<0.05, **P<0.01.

For the rTPJ region, one-sample t-tests revealed significant interpersonal brain synchronizations in the band between 0.02 and 0.08Hz in CH14 (t(50)=3.04, P=0.004) and CH5 (t(50)=2.93, P=0.005) for FF and in CH19 for FB (t(45)=3.42, P=0.001) (FDR corrected) (Figure 3C). Further, independent-samples t-test (uncorrected) showed that only CH14 (t(95)=2.05, P=0.04, Cohen’s d=0.41) had a higher synchronization in FF compared to FB (Figure 3D). Neither CH5 (t(95)=1.61, P=0.11, Cohen’s d=0.33) nor CH19 (t(95)=−0.77, P=0.45, Cohen’s d=0.16) showed such a difference. To test detailed effects in CH14, mixed ANOVAs on coherence measures showed no significant main effect of Time (F(2,190)=2.15, P=0.12, η_p²=0.02) and no interaction effect of Condition×Time (F (2,190)=0.18, P=0.84, η_p²=0.002). Moreover, coherence measure in CH14 was positively correlated with shared intentionality for FF (SIR: r=0.28, P=0.046), but not for FB (SIR: r=0.02, P=0.89). No significant correlations were found in any other channels (Figure 4). Finally, no significant synchronizations were found in rDLPFC channels for both FF and FB, neither for the one-sample t tests (ps>0.09, FDR corrected) nor for the independent-samples t-tests (ps>0.1, uncorrected) (Figure 3C and D).

Open in a separate window

Fig. 4.

Correlations. (Left) Correlation r-maps of synchronization indexed by increased coherence and shared intentionality rate among all channels for FF and FB interactions. (Right) The correlation between interpersonal brain synchronization and shared intentionality rate in the rTPJ (CH14) was only significant in the FF but not in FB (*P<0.05).

Discussion

In this study, we investigated how FF interaction impacts interpersonal brain synchronization during two-person economic exchange by combining an economic exchange game paradigm with fNIRS hyperscanning. Pairs of strangers interacted repeatedly either FF or FB while their brain activations were simultaneously recorded in rTPJ (target region) and rDLPFC (control region). Our findings revealed that FF interactions increased shared intentionality, the positive belief of cooperative decisions of each other and facilitated actual cooperation between partners compared to face-block interactions. During FF interaction, the responder’s brain activity in rTPJ region decreased when the proposer was more honest and offered more proportion, and increased when the responder made more rejections. The interpersonal brain synchronization in rTPJ (but not in rDLPFC) was greater during FF compared to FB interactions and interpersonal brain synchronization in rTPJ increased with shared intentionality between partners.

In everyday life, uncertainties about other people’s intention influences interpersonal interactions. Previous research has shown that FF interaction reduces such uncertainty by providing social cues (i.e. external visible cues such as facial expression or body gestures) (Okdie et al., 2011). FF interactions allows people to share psychological states (Gilbert, 1989; Searle, 1995; Tuomela, 1995), understand each other better (Bordia, 1997) and develop a better mental model about each other (Hill et al., 2009). Our behavioral results support these earlier findings by showing that FF compared to FB interactions elicited greater shared intentionality between partners resulting in less rejection and more quick intention judgment; therefore, facilitating cooperation such that proposers were more likely to infer that responders would accept their offer and responders were more likely to believe that proposers told the truth. During FF interactions, both players probably used external visible cues such as facial expression and body gestures to infer their partner’s psychological states (i.e. intentions, beliefs, goals), which resulted into appropriate decisions over time leading to an increased cooperation and higher earnings for both players. In addition, the correlation between shared intentionality and other behavioral measurements in both conditions confirmed that the differences between the two conditions could not have been explained by lack of engagement of the players, and highlight the effect of FF interaction.

We did not find any differences in the deceptive and unfair behavior between conditions. Previous studies that comparing honesty deception behavior in FF vs. computer-based interaction find inconsistent results. On the one hand, some studies show that partners in FF compared to computer-mediated and written interactions elicit less deception because of greater social or normative pressure in FF interaction (Bordia, 1997; Valley et al., 1998). On the other hand, another study has found greater deceptive behavior in FF than in computer-mediated communication for the availability of rich cues and proactive action based on partner’s feedback (Lewis and George, 2008). In our study, deceptive and unfair strategies were discouraged independently of the interaction condition, since both the proposer and responder would have found out about the true outcome at the end of each round of the game. In addition, post measurements of emotions did not show difference between conditions. It is not easy to estimate the effects of emotion on the observed interactions, since pairs interacted repeatedly many times and might have different kinds of emotions during different trials.

On the neural level, responders’ activity in rTPJ decreased with proposers’ honesty, offers, and increased with responders’ rejection in FF interaction, indicating the role of FF interaction in reducing uncertainty. Specifically, pairs interacting FF showed higher interpersonal brain synchronization in rTPJ than those interacting FB. Only during FF interaction synchronization in rTPJ increased with greater shared intentionality between partners. Previous studies have found that TPJ establishes a social context for behavior by being a convergence zone for attention, memory, language and social cognition (Carter and Huettel, 2013). Moreover, TPJ plays an unique and independent role in processing social information about future behavior (Carter et al., 2012). Furthermore, a rich body of evidence indicates that rTPJ is involved in mentalizing (Frith and Frith, 2003), reasoning others’ desires and beliefs (Saxe, 2006), and representing others’ intention (Koster-Hale et al., 2013). The observed interpersonal brain synchronization in rTPJ might be an indicator of shared intentionality between partners. Using external cues from each other, partners were able to better infer the other person’s intentions to build a correct mental model about each other, resulting into greater cooperation during the economic exchange. Besides mentalizing, TPJ has also a more general function in attentional orientation (Decety and Lamm, 2007; Mitchell, 2008). For example, previous fMRI studies have found that rTPJ mediates joint attention between people during FF interactions (Redcay et al., 2010). Therefore, the increased interpersonal brain synchronization in rTPJ might just indicate an alignment of joint attention during FF interaction. However, such an attentional account could not explain the correlation in rTPJ between interpersonal brain synchronization and shared intentionality. In addition, a previous EEG hyperscanning study (Babiloni et al., 2007b) that required card players to indicate the card by voice has found neural connection in brain regions (prefrontal and anterior cingulated cortex) different from our study, indicating that voice report by proposer only in FF condition might not explain the neural differences in our study.

In contrast, no interpersonal brain synchronization differences were found in rDLPFC, a region consistently shown to be engaged in strategic decision-making (Miller and Cohen, 2001), e.g. with including cognitive control and goal maintenance during economic exchange (Miller and Cohen, 2001; Sanfey et al., 2003; Knoch et al., 2009). Further, an EEG hyperscanning study showed synchronized neural oscillations in right frontocentral regions employing the standard ultimatum game, in which behaviors of players were modulated in FF interaction (Yun et al., 2008). However, in our task, we did not find differences in deceptive and unfair behaviors comparing FF with FB condition, indicating that different interaction styles did not influence strategic decision-making; and, therefore, diminished interpersonal brain synchronization in rDLPFC. Overall, the precise role of the rDLPFC in human FF interaction is of great interest and requires additional investigation in future studies.

There are a couple of limitations for the present study. First, we cannot directly exclude the effect of voice reporting on the synchronization in rTPJ based on shared intentionality, although this region has been previously reported unrelated to voice report during social interaction and joint attention (Redcay et al., 2010) as well as in a card game with voice report (Babiloni et al., 2007b). Second, we cannot dismiss the potential effects of emotion on the establishment of shared intentionality during the economic exchange, since emotion plays an important role during cooperation (Wubben et al., 2009) and bargaining (Lelieveld et al., 2012). Future studies are needed with specific emotional priming or dynamic emotional measurement during the social interactions to provide direct evidence.

In summary, our study revealed an interpersonal brain synchronization in rTPJ during naturally occurring FF economic exchange, confirming earlier evidence that this region is engaged in inferring others’ mental states such as goals, intentions, and desires (Overwalle, 2009; Overwalle et al., 2009). FF interaction plays not only a unique role in human’s ontology but also in human’s phylogeny. For example, neonates and infants show innate preferences for faces and develop gradually intra- and interpersonal coordination during FF communication (Frith and Frith, 2003; Schilbach et al., 2012). Future studies on the developmental perspective of naturally occurring FF interactions and its underpinning neural signatures that combine fNIRS hyperscanning with social exchange paradigms are warranted, especially for clinical investigations that target social brain disorders such as autism spectrum disorders.

Funding

This work was supported by the National Basic Research Program of China (2011CB711000;, 2013CB837300), the National Natural Science Foundation of China (NSFC) (31170971;, 61210010), and the Major Project of National Social Science Foundation (12&ZD228) to C.L. and the Major Project of National Social Science Foundation (13&ZD155), Humanities and social science projects supported by Ministry of Education (13YJA190007), the National Natural Science Foundation of China (NSFC) (31400888), CNLYB1212 to X.M.

Conflict of interest. None declared.

References