Breaking Harmony Square: A game that “inoculates” against political misinformation

---

Implications

Harmony Square is an interactive social impact game about election misinformation.¹The game can be played in any browser at https://www.harmonysquare.game The goal of the game is to reveal the tactics and manipulation techniques that fake news producers use to mislead their audience, build a following, and exploit societal tensions to achieve a political goal. The game’s setting is Harmony Square, a peaceful place where residents have a healthy obsession with democracy. At the start of the game, players are hired as Chief Disinformation Officer. Their job is to ruin the square’s idyllic state by fomenting internal divisions and pitting its residents against each other, all while gathering as many “likes” as they can. In order to deliver sufficiently weakened doses of the informational “virus,” Harmony Square makes use of humor throughout the game. For example, players can share humorous messages in a fictional social network, and are shown entertaining headlines in a news ticker at the top of the screen (see the second panel in Figure 1). Aside from increasing the entertainment value of the game, the use of humor in inoculation interventions has the added benefit of potentially decreasing reactance, i.e., resistance to voluntarily engaging with the intervention (Compton, 2018; Vraga et al., 2019). Over the course of 4 different levels (Trolling, Emotion, Amplification and Escalation), the player’s misinformation campaign causes the square to gradually go from a peaceful state to full-blown mayhem. Figure 1 shows a number of screenshots of what the game’s landing page and game environment look like.

The game was produced with and based on the US Department of Homeland Security’s Cybersecurity and Infrastructure Agency’s (CISA) approach to understanding foreign interference in elections. In an effort to “strengthen the national immune system for disinformation,” CISA launched a campaign in 2019 that sought to expose common misinformation tactics through the lens of an ostensibly innocent and non-partisan issue: whether or not to put pineapple on pizza (Ward, 2019). In a similar vein to the “pineapple pizza” campaign, Harmony Square exposes the 5 steps of the election misinformation playbook: targeting divisive issues (in line with the “trolling” & “emotion” scenarios in Harmony Square, in which players learn how to turn an ostensibly neutral issue into a heated and polarizing debate); moving accounts into place; amplifying and distorting the conversation (the “amplification” scenario, in which players learn how bots and fake accounts can be used to amplify the reach of manipulative content); making the mainstream; and taking the conversation into the real world (the “escalation” scenario, in which players learn how to escalate online debates into real-world action) (CISA, 2019).

Harmony Square builds on the success of our other gamified anti-misinformation interventions, such as the Bad News game (Roozenbeek & van der Linden, 2019; Roozenbeek, van der Linden, et al., 2020).²The Bad News game can be played at https://www.getbadnews.com Similar to Bad News, playing Harmony Square builds cognitive resistance against common forms of manipulation that people may encounter online by preemptively warning and exposing people to weakened doses of these techniques in a controlled environment. Unlike Bad News, Harmony Square is an election game that focuses specifically on how misinformation can be used to achieve political polarization (Bessi et al., 2016; Shao et al., 2017), for example, by fueling outgroup hostility, a critical element of both organic misinformation and targeted disinformation campaigns, particularly during contentious political events such as the 2020 US Presidential elections (Groshek & Koc-Michalska, 2017; Hindman & Barash, 2018; Iyengar & Massey, 2018; Keller et al., 2020). 

The idea of psychological “vaccines,” coined in the 1960s by the psychologist William McGuire (McGuire, 1964; McGuire & Papageorgis, 1961b), is called inoculation theory. It follows a medical analogy: a vaccine is usually a weakened version of a particular pathogen which, after being introduced to the body, induces the production of antibodies, preventing an individual from becoming sick when exposed to the real illness. Inoculation theory states that the same can be achieved with (malicious) persuasion attempts: pre-emptively exposing someone to a weakened version of a particular misleading argument prompts a process that is akin to the production of “mental antibodies,” which make it less likely that a person is persuaded by the “real” manipulation later on (Compton, 2013; van der Linden et al., 2017). During gameplay, players are exposed to weakened doses of manipulation techniques by stepping into the shoes of a fake news producer to trigger the production of psychological antibodies.

Instead of focusing on specific examples, also known as issue-based inoculation (which has been the standard in inoculation research), Harmony Square builds cognitive resistance against the techniques that underpin a whole range of political misinformation in an attempt to achieve broad-spectrum resistance against manipulation (Basol et al., 2020; Cook et al., 2017). The game functions as a perspective-taking exercise (i.e., by putting the player in the position of a fake news creator), an approach known as “active inoculation” (McGuire & Papageorgis, 1961a; Roozenbeek & van der Linden, 2018). Actively involving individuals in the inoculation process by playing a game—as opposed to subjecting them to a more passive reading exercise—has the potential advantage of increasing retention in memory and increasing the longevity of in the inoculation effect (Compton & Pfau, 2005; Maertens et al., 2020; Roozenbeek & van der Linden, 2019).

By playing through the 4 levels in Harmony Square, players learn about 5 manipulation techniques, all of which are common features of political misinformation (van der Linden & Roozenbeek, 2020): 

1. Trolling people, i.e., deliberately provoking people to react emotionally, thus evoking outrage see (McCosker, 2014; Roozenbeek & van der Linden, 2019).
2. Exploiting emotional language, i.e., trying to make people afraid or angry about a particular topic (Brady et al., 2017; Zollo et al., 2015).
3. Artificially amplifying the reach and popularity of certain messages, for example through social media bots or by buying fake followers (McKew, 2018; Shao et al., 2017).
4. Creating and spreading conspiracy theories, i.e., blaming a small, secretive and nefarious organization for events going on in the world (Lewandowsky et al., 2013; van der Linden, 2015).
5. Polarizing audiences by deliberately emphasizing and magnifying inter-group differences (Iyengar & Massey, 2018; Prior, 2013).

In this study, we find that people who played Harmony Square rated manipulative social media posts making use of the above techniques as less reliable after playing, were more confident in their ability to spot such content, and importantly, were less likely to report to share it in their social network. This finding held for both “real” manipulative content that has gone viral online in the past, as well as for “fictional” content that participants in our study had never seen before. These findings highlight the potential for social impact games as an effective approach to counter misleading, fake, or manipulative content proliferating online (Basol et al., 2020; Maertens et al., 2020; Roozenbeek, Schneider, et al., 2020).

In addition, we find that political ideology did not interact with the learning effect conferred by Harmony Square, meaning that the game was effective at teaching manipulation techniques for both liberals and conservatives. This finding that the intervention can be effective across partisan lines is particularly important in light of the polarization of not just the US media landscape, but of the term “fake news” itself (van der Linden et al., 2020).

Specifically, we tested if people became better at spotting troll posts (i.e., posts that “bait” people into responding emotionally), exploitative emotional language use, conspiratorial content, and content that deliberately seeks to polarize different groups. These are all important manipulation techniques used in political misinformation (Roozenbeek & van der Linden, 2019; van der Linden & Roozenbeek, 2020). In addition, these techniques are key components of many organized disinformation campaigns (Bertolin et al., 2017; Cook et al., 2017; Hindman & Barash, 2018). Building cognitive resistance against these techniques at scale is a powerful tool to reduce the risk of disinformation campaigns affecting the democratic process. Recent research with the Bad News game has looked into the longevity of the inoculation effect conferred by “fake news” games such as Bad News (Maertens et al., 2020). This research finds that significant inoculation effects remain detectable for at least one week, and much longer when participants are presented with very short reminders or “booster shots.” 

In short, we show that Harmony Square is effective at reducing some of the harmful effects that manipulative content can have on individuals. Taking only around 10 minutes to complete, the game conveys critical information about key manipulation techniques in a fun and interactive manner.We note that Harmony Square focuses mostly on political misinformation, and the game’s setting is a fictional democratic society. Since (political) misinformation is a significant problem in non-democratic countries, one of the limitations of this game is its limited applicability in countries that lack free elections. In addition, although research has shown that gamified inoculation effects can persist for months (Maertens et al., 2020), the longevity of the effect of Harmony Square was not evaluated here. 

Findings

The purpose of this study was to find out whether people who play Harmony Square 1) find manipulative social media content less reliable after playing; 2) are more confident in their ability to spot such manipulative content; and 3) are less likely to indicate that they are willing to share manipulative social media content in their network, compared to a gamified control group (which played Tetris for around the same amount of time it takes to complete Harmony Square). To answer these questions, we first calculated the difference between the average score on all 3 of these questions for all 8 “real fake news” and all 8 “fictional fake news” social media posts that we used as measures (see the “methods” section) before and after the intervention, for each participant. We call this the “pre-post difference score.” We then checked if these difference scores for the treatment group were significantly different from the control group for each outcome variable, using an Analysis of Variance (ANOVA).

Finding 1: People who play Harmony Square find manipulative social media content significantly less reliable after playing compared to a control group.

For all the “real fake news” social media posts combined (see “methods” section), we find a significant main effect of the treatment condition (i.e. playing Harmony Square) on aggregate reliability judgments, meaning that playing the game significantly reduces the perceived reliability of “real fake news” compared to the control group (F(1,679) = 43.21, p < .001, η² = .060, d = 0.51, Figure 2).³Specifically, compared to the control condition, the shift in post-pre difference scores for reliability judgments was significantly more negative for the treatment group (M_diff,control = -0.16, SD_diff,control = 0.66 vs M_{diff,treatment} = -0.52, SD_{diff,treatment} = 0.77, d = 0.51). We found no main effect (F(2,675) = 0.486, p = 0.62) nor an interaction effect (F(2,675) = 0.663, p = 0.52) for political ideology, meaning that ideology does not make a significant difference to the inoculation effect. We find the same result for the “fictitious fake news” items (F(1,679) = 48.79, p < .001, η² = .067, d = 0.54).⁴Similar to “real” fake news, the shift in post-pre difference scores for reliability judgments was significantly more negative for the treatment group (M_diff,control = -0.095, SD_diff,control = 0.55 vs M_{diff,treatment} = -0.44, SD_{diff,treatment} = 0.72, d = 0.54). Here, we also found no main effect (F(2,675) = 1.154, p = 0.32) nor an interaction effect (F(2,675) = 0.810, p = 0.45) for political ideology. Importantly, the effect-sizes are nearly identical, which illustrates that using real or fictitious fake news does not matter much for assessment. The results are visualized in Figure 2.

Finding 2: People who played Harmony Square are significantly more confident in their ability to spot manipulative content in social media posts, compared to a control group.

Next, we checked if playing Harmony Square increases people’s confidence in spotting manipulative content. We used the same method as above: first, we calculated the difference between the average scores before and after the intervention for both the treatment and control group, and then conducted an ANOVA to see if the differences between treatment and control were significant. Participants who played Harmony Square became significantly more confident in their ability to spot both “real” (F(1,679) = 14.52, p < .001, η² = .021, d = 0.30)⁵Compared to the control condition, the shift in post-pre difference scores for confidence judgments was significantly more positive for the treatment group (M_diff,control = -0.051, SD_diff,control = 0.55 vs M_{diff,treatment} = 0.15, SD_{diff,treatment} = 0.82, d = 0.30). and “fictional” (F(1,679) = 14.55, p < .001, η² = .021, d = 0.30)⁶Compared to the control condition, the shift in post-pre difference scores for confidence judgments was significantly more positive for the treatment group (M_diff,control = -0.011, SD_diff,control = 0.59 vs M_{diff,treatment} = 0.19, SD_{diff,treatment} = 0.79, d = 0.30). manipulative content. Figure 3 visualizes the results.

Finding 3: People who played Harmony Square are significantly less likely to report sharing manipulative social media content with others.

Finally, we checked whether playing Harmony Square reduces participants’ self-reported willingness to share manipulative content with people in their network. Using the same method as above, we found that people who played the game were significantly less likely to share both “real” (F(1,679) = 12.85, p < .001, η² = .019, d = 0.28)⁷Compared to the control condition, the shift in post-pre difference scores for willingness to share was significantly more negative for the treatment group (M_diff,control = -0.09, SD_diff,control = 0.53 vs M_{diff,treatment} = -0.25, SD_{diff,treatment} = 0.64, d = 0.28). and “fictional” (F(1,679) = 12.62, p < .001, η² = .018, d = 0.27)⁸Compared to the control condition, the shift in post-pre difference scores for willingness to share was significantly more negative for the treatment group (M_diff,control = -0.06, SD_diff,control = 0.51 vs M_{diff,treatment} = -0.20, SD_{diff,treatment} = 0.56, d = 0.27). manipulative content that they encounter online. Figure 4 visualizes the results.

To restrict multiple testing, we only present results for the aggregated fake news indices here. However, when looking at the manipulation techniques featured in the game (trolling, emotion, conspiracy and polarization), we show that players improve on each technique as well. The full list of ANOVAs per manipulation technique, including effect size estimates, can be found in Supplementary Table S3. A bar plot which summarizes the results for the pre and post-test separately in a single figure can be found in Supplementary Figure S1. We conducted two robustness checks to verify the main analyses presented here: a linear regression using post-test as the dependent variable, pre-test as a covariate, and condition as a between-subject factor (see the Supplementary Methods and Analyses section and Supplementary Table S4); and a robust linear regression clustering scores at the participant and rating level (see the Supplementary Methods and Analyses section and Supplementary Table S5). Both approaches give the same results as what is presented above.

Methods

To test if Harmony Square improves people’s ability to spot manipulative online content, we conducted a 2 (treatment vs control) by 2 (pre vs post) mixed design randomized controlled trial.⁹The full dataset, “real” and “fictional” social media posts and R scripts used in this study are available on the OSF. Link: https://osf.io/r89h3/. The treatment condition involved playing Harmony Square from beginning to end. The control condition played Tetris for about 10 minutes. We chose Tetris because it is in the public domain, most people know how it works without practicing, and it involves about the same amount of cognitive effort as playing Harmony Square. Following the methodology established in prior research on “fake news” games (Basol et al., 2020; Maertens et al., 2020; Roozenbeek, Maertens, et al., 2020; Roozenbeek & van der Linden, 2019), we measured reliability judgments of social media posts containing misinformation, both before and after the intervention on a 1-7 Likert scale. 

Measures

We sought to answer three questions about the effectiveness of Harmony Square as an anti-misinformation tool: 

1. Does playing Harmony Square make people better at spotting manipulation techniques commonly used in political misinformation?
2. Does playing the game increase people’s confidence in their ability to spot such manipulation techniques in social media content?
3. Does playing the game reduce people’s self-reported willingness to share manipulative social media content with people in their network? 

To address these questions, we showed the participants in our study 16 social media posts, each of which made use of one of 4 manipulation techniques learned while playing Harmony Square: trolling, using emotional language, conspiratorial reasoning, and group polarization. These posts were selected to be a mix of politically partisan and politically neutral content. Politically neutral items covered topics such as a kidnapping at an amusement park. Since the Harmony Square game is about political misinformation, we also included several items that were ideologically or politically charged. These items were balanced overall, with an equal number of right-leaning and left-leaning items. All items are available on our OSF page as well as in supplementary table S6.

In total, 8 of these posts were examples of “real” manipulative content found “in the wild” on social media and in fake news articles. The other 8 were social media posts that we created (“fictional fake news”), which were validated in previous research (Basol et al., 2020; Maertens et al., 2020; Roozenbeek, Maertens, et al., 2020). We did not hypothesize any significant differences between participants’ assessments of “real” and “fictional” misinformation, but chose to include both types for the following reasons: 1) including “real” items increases the ecological validity of the study, as participants are tested on information that they could have encountered “in the wild”; 2) including “fictional” items maximizes experimental control and thus allows us to better isolate each manipulation technique and ensure political neutrality, and 3) by including “fictional” items, we account for the possibility that participants may have seen the “real” manipulative content before, a memory confound which could bias their assessment (Roozenbeek & van der Linden, 2019).¹⁰The Supplementary Methods & Analyses appendix contains further information on the item selection procedure and a Principal Component Analysis for both the real and fake social media posts. Following Basol et al. (2020), we deliberately chose to only include manipulative content, as opposed to a mix of manipulative and non-manipulative content. The purpose of the Harmony Square game was not to learn how to distinguish high-quality and low-quality content, but rather to teach people how to spot common types of misinformation on social media. We therefore chose to focus on addressing the question of whether Harmony Square is effective at reducing susceptibility to political misinformation, rather than truth discernment (Pennycook et al., 2020) but we note that media literacy interventions can affect the rating of both credible and non-credible items (Guess et al., 2020). For a more detailed discussion on how gamified inoculation interventions affect people’s perception of “real” (high-quality) news we refer the reader to Roozenbeek, Maertens et al. (2020).

Figure 5 shows an example of what our items look like in the survey environment. The social media post in the figure is a real example of a rumor that went viral about a girl being kidnapped in a theme park, an example of the “emotional language” technique learned in the game (AFP Canada, 2019; Pennycook et al., 2019). The full list of items can be found on the OSF¹¹See https://osf.io/r89h3/ and in the supplement (Table S6). 

For each of the 16 items, we asked participants 3 questions, which they could answer on a 1-7 scale (1 being “not at all”, 4 being “neutral” and 7 being “very”):¹²A reliability analysis shows acceptable to good internal consistency for all 3 outcome measures, both for the “real” fake news item set and the “fictional” fake news item set (M_{real,reliability} = 3.26, SD_{real,reliability} = 0.98, α = 0.68; M_{fake,reliability} = 2.99, SD_{fake,reliability} = 1.06, α = 0.78; M_{real,confidence} = 5.20, SD_{real,confidence} = 1.06, α = 0.84; M_{fake,confidence} = 5.13, SD_{fake,confidence} = 1.08, α = 0.86; M_real,sharing = 2.26, SD_real,sharing = 1.17, α = 0.85; M_fake,sharing = 2.15, SD_fake,sharing = 1.19, α = 0.88).

1. How reliable do you find this post?
2. How confident are you in your judgment?
3. How likely are you to forward this post to others?

We asked these questions for all 16 social media posts both before and after the intervention (Harmony Square for the treatment group, and Tetris for the control group). This allowed us to measure the difference between the “before score” and the “after score” for each group (the “pre-post difference score”). We thus arrive at our hypotheses: if Harmony Square is effective as an anti-misinformation tool, participants who played it should 1) find manipulative content significantly less reliable after playing, 2) be significantly more confident in their judgment, and 3) be significantly less likely to report sharing such content with people in their network, whereas the control group—who did not learn anything about manipulative content while playing Tetris—should show no significant differences for each of the three questions before and after playing. To control for multiple testing, we only evaluated the aggregate indices for each dependent variable but for completeness we report effects for all 4 manipulation techniques separately in the supplementary information (Supplementary Table S3). Descriptive statistics for each individual item can be found in Supplementary Table S2.

In total, 681 people were recruited in 2 separate data collections; a US-only sample (n = 312) and an international sample (n = 369). We pooled the results here (effect-sizes are slightly larger for the US-only sample). In total, 296 participants played Harmony Square (the treatment group), and 385 people played Tetris (the control group). A detailed overview of the sample selection process and study participants, as well as several robustness checks for the main analyses, can be found in the Supplementary Methods & Analyses appendix.