Combatting online hate: Crowd moderation and the public goods problem

Low-hate online discussions as a public good

The classic examples of public goods in economic theory, like fish in the sea and access to blood transfusions (Abásolo and Tsuchiya, 2014; Cornes and Sandler, 1996, p. 9), are tangible and desirable, but the concept can be equally valuable in fostering understanding of a good like access to low-hate online discussions. While individuals may have diverging views on what constitutes a desirable and pleasant debate environment (Duggan, 2017; Gillespie, 2018), we specify low-hate as resembling a universally desirable trait since social media users generally prefer restrictions on extreme speech on social media (Rasmussen, 2024). Additionally, the public goods analogy does not rely on preference symmetry (Cornes and Sandler, 1996, p. 241).

It can be illustrative to think of low-hate online discussions as we think of voluntary blood banks as a public good given that both feature the core characteristics of public goods. One person receiving a blood transfusion does not (meaningfully) restrict other people’s access to treatment (non-rivalness) (Cornes and Sandler, 1996), and one person enjoying an online debate experience does not take away from other people enjoying it. In addition, access to blood transfusions and low-hate online discourse feature non-excludability, as individuals can enjoy the good without contributing to its production (Abásolo and Tsuchiya, 2014; Cornes and Sandler, 1996, p. 9). Voluntary blood donation schemes rely on many individuals providing a solution to a problem (illness) by donating their private resources (blood). Similarly, the “production” of a pleasant, low-hate online debate environment involves protecting online discussions from “pollution” by hateful rhetoric (problem) through donating time (private resource) to engage in moderation that remedies (solution) the negative influence of hate in online discussions (Gillespie, 2018). Combined, these features of public goods can be expected to produce negative externalities, as they create incentive structures that promote freeriding, leading to an undersupply of valuable, public goods (Cornes and Sandler, 1996, p. 539). If one can enjoy the pleasant online discussion without contributing to moderation, one has limited incentive to help keep online discussions “on track.” Note that in the context of online discussions there are two levels of freeriding: People can be “lurkers” who consume online discussions without contributing to them by posting, and people can enjoy a pleasant online discussion without contributing to crowd moderation. We focus on the latter type of freeriding (which can be considered a “second-order public good,” Fehr and Gächter, 2002; Axelrod, 1986).

Varied costs of online bystander reactions

Expecting respondents to show reluctance to engage in online bystander reactions because of incentives to freeride begs the question: What are the costs and benefits of reacting when witnessing hate online?

Some costs cut across all types of bystander reactions (Cepollaro et al., 2023), including the time and effort spent evaluating whether content can be deemed hateful and requires reaction, the time and energy it takes to engage in a reaction (press a series of buttons, compose a message, etc.), and the digital literacy required to navigate response options (Crawford and Gillespie, 2016; Obermaier, 2022). Other costs include the negative emotions associated with thoroughly reading a hateful statement and contemplating a reaction (Duggan, 2017; Vogels, 2021; Woods and Ruscher, 2021) and potential damage to social relationships when one engages in heated discussions (Carlson and Settle, 2022).

The literature on online bystander reactions and civil intervention frequently discusses costs in relation to “thresholds” (Hansen et al., 2023; Kunst et al., 2021; Porten-Cheé et al., 2020). Like the least costly response option, do nothing, low threshold reaction options take little effort and are typically enabled by platform tools (Kunst et al., 2021, p. 259; Porten-Cheé et al., 2020, p. 520), e. g., downrating or reporting content to platform moderators. Reporting content as hateful is private and only relays information to platform moderators. High threshold options, e. g., counter-speech in response to hateful content (Kunst et al., 2021, p. 259; Porten-Cheé et al., 2020, p. 521), take more effort and are more costly than reporting because they are publicly visible, also to authors of hateful statements (Kunst et al., 2021, p. 260). The visibility aspect entails potential social repercussions from (un)known spectators (Carlson and Settle, 2022) and makes one a target of future hateful outbursts (Aldamen, 2023; Johnson, 2018, p. 1285; Kutlaca et al., 2020).

We expect higher individually experienced costs to reduce the motivation to assist in crowd moderation, which leads to our second hypothesis:

H2: High-threshold bystander reactions such as “reply” will be less frequent compared to low-threshold reactions such as “report,” such that reporting will be the most frequent behavior, followed by sharing, followed by replying.

One straightforward and high affiliated cost for an individual user is encountering a large amount of online hate. We expect witnessing more online hate to increase motivations to reduce hate through bystander reactions (reporting, counter-speech). As users witness more hate, they also experience the cost of reaction as higher and would be better off remaining passive (Obermaier, 2022, p. 11). However, at some point, the effect should wear off as individuals become too burdened. Two considerations lead us to believe this.

First, up to a point, exposure to online hate indirectly informs individuals of the problem at hand. Witnessing low to medium amounts of hate lets individuals perceive a low-hate online debate environment as a beneficial good worth co-creating and thus to view hateful content as a risk factor.^[3] This might lead some to engage in prosocial bystander reactions to attain the good, much like how an increased need for blood transfusions in the aftermath of natural disasters might motivate otherwise passive citizens to temporarily donate. Yet, at high levels of exposure to online hate, the perceived benefits of a pleasant debate environment are likely outweighed by the aggregated negative emotions affiliated with exposure, combined with the increased experienced costs of reacting as the individual observes the harsh debate tone.

Second, exposure to high levels of online hate also raises the individual bystander’s awareness of others’ inactivity, which means limited exposure to counter-speech. This experience may mean that users perceive crowd moderation as a particularly costly act (I will be on my own) and benefits as unlikely to materialize (I cannot overcome this issue alone). We summarize our expectations related to exposure to online hate in our third hypothesis:

H3: A non-linear (inverted u-shape) relationship such that overall (mean) exposure to other-directed online hate correlates positively with overall (mean) prosocial bystander reactions, but there will be a drop at high levels of exposure.

Data

Respondents were invited to the survey through e-Boks, a digital mailbox all permanent Danish residents (age 16+) are required to have. The invitation, “Help researchers to understand hate on social media,” contained information about the purpose of the study and was sent to a representative sample of adult Danes based on a sample of 300,000 individuals drawn by Statistics Denmark. Some 16,026 individuals from this sample had no valid account at the time of the study, leaving a net sample of 283,974. We received 24,996 responses, equaling a response rate of approximately 9 %, exceeding our preregistered minimum sample size (10,000 respondents). Sample size was determined not as a function of power to detect a certain effect size, but rather with the goal of using a large-scale, representative sample of the Danish population. No participants met the preregistered exclusion criteria. However, since the survey allowed participants to skip questions they did not want to answer, there are missing values for a sizable portion of participants, and the number of observations differs between statistical tests. The relatively low response rate is not surprising considering that no reminders were sent, and considering expected survey fatigue following the COVID19-pandemic. Still, we are confident that the overall data patterns reported are generalizable to the Danish population. We supplemented the survey data with data on participants’ sex from the Danish national registries.

Measures

The two main variables are respondents’ self-reported exposure to online hate and self-reported reactions to the exposure. To measure exposure to online hate, we presented respondents with three question batteries. The first asked “How often have you personally been the target of hateful comments on …” and listed 9 widely used social media platforms. For each platform, respondents could answer very often, often, once in a while, rarely, and never. The second and third battery used the same list of social media platforms and answer categories but asked how often the respondent had experienced “people you know” and “people you don’t know,” respectively, being targets of hate online. We created two overall indices of how much hate, self- or other-directed, a respondent is exposed to across their social media usage by multiplying experienced hate for each platform by frequency of usage, summing for all platforms the person indicated using. The indices range from 0 to 128, have a mean of 1.38 (self-directed hate) and 7.64 (other-directed hate), and a standard deviation of 4.22 (self-directed hate) and 8.74 (other-directed hate). Since these indices are heavily right-skewed, we followed our preregistered protocol and log-transformed them for most of our analyses (see Appendix D of the supplementary material for model results with untransformed scores). When online hate targets a group, e. g., an ethnic minority, the distinction between self- and other-related hate may be less straightforward, as individuals may infer self-targeting based on group membership, something our operationalizations do not allow us to analyze.

To measure reactions to online hate, including crowd moderation efforts, respondents were asked one or two batteries of questions depending on their indicated exposure to online hate. All respondents who indicated having experienced self- and/or other-directed online hate were asked how often they, in those situations, had “reacted in the following ways.” Possible reactions were: (1) “It made me sad”; (2) “It made me angry”; (3) “It made me scared”; (4) “I responded to the comment”; (5) “I shared the comment with others either on or off social media”;^[4] (6) “I reported the comment for violating the guidelines of the social media platform”; (7) “It reduced my desire to participate in online debates”; and (8) “It increased my desire to participate in online debates.” For each item respondents could answer very often, often, once in a while, rarely, and never. Items 1–3 measure emotional reactions; items 4–6 measure crowd moderation efforts; items 7–8 measure intentions to withdraw from online discussions due to exposure to online hate. The distinction between hate targeting “people you know” and “people you do not know” was left out for the measurement of reactions to hate due to the length of the survey. Appendix B lists the range, distribution, mean and standard deviation for all variables, including reaction items.

In addition to the two main variables of exposure and reactions to online hate, we measured several covariates. Following the preregistered expectations, we investigate three covariates: individual personality traits, perceptions of political discussions online, and a measure of “core human values.” A detailed description of these measures and related analyses is available in Appendix D. Full wordings of all questions and answer categories can be found in Appendix A.

Analysis plan

All analyses were carried out using R (version 4.3.2 for Mac OS 14.3). As the distributions of answers to the reaction to online hate items focused on crowd moderation efforts (report, answer, share) were non-normally distributed, we use nonparametric tests where appropriate.

Deviations from preregistration

We deviate from the preregistration in a few instances. We have changed the naming protocol of hypotheses to fit the structure of the article (see Appendix C for an overview of differences in naming protocols). To streamline the terminology used throughout the article, we changed mentions of hate in preregistered hypotheses to “online hate.” Additionally, we had erroneously proposed to analyze differences in reaction frequencies (Hypothesis 4) as between-subject type analyses in the preregistration (Dunn’s all-pairs rank comparison test). Since these differences are within-subject, we instead conduct paired Wilcoxon tests. Finally, we preregistered six additional hypotheses besides the five discussed in the main text. These are presented and analyzed in Appendix D. We have also included two descriptive hypotheses that did not appear in the preregistration (H1 and H4).

Figure 1:

Average exposure to hate towards the self, known others, and unknown others on different social media platforms.

Note: Mean values for self-reported exposure to hateful content directed at unknown others, known others, or oneself on different social media platforms. Error bars (where visible) denote standard error of the mean. Answers were given on a 5-point Likert scale (0 = never, 4 = very often).

Descriptive statistics

Overall, participants reported to be exposed to hate on most social media platforms to a considerable degree, although the frequency of exposure varies across platforms and target of hate (see Figure 1).

Also, common responses to hateful content vary widely, depending on whether the experienced hate is self- or other-directed. The most common response to hate regardless of target is to consider withdrawing, followed by anger, and sadness (see Figure 2).

Figure 2:

Average reactions when exposed to hate towards self or others.

Note: Mean values for self-reported reactions to hateful content directed at others (known or unknown) or oneself. Error bars are too small to be seen in this plot. Answers were given on a 5-point Likert scale (0 = never, 4 = very often).

Inferential statistics

In line with H1, we find that for reporting, answering, or sharing hateful content, the most common reaction type is to remain passive with the never option as most frequently chosen by participants both in terms of self- and other-directed hate (see Figure 3 and Table D.3 in Appendix D). Respondents were more prone to abstain from reacting (answer, report, share), i. e., answer never, when the hate was other-directed (answer: 71 %, report: 72 %, share: 80 %) than when it was self-directed (answer: 31 %, report: 58 %, share: 59 %).

H2 predicted that in terms of different prosocial reactions to hate (answering, reporting, sharing), high-threshold reactions (i. e., answering) would be less common than low-threshold reactions (i. e., reporting). For responses to other-directed hate, reflecting classic bystander situations, reporting (M = 0.55, SD = 1.02) was indeed the most frequent action, followed, contrary to our hypothesis, by answering (M = 0.44, SD = 0.79; reporting vs. answering Wilcoxon signed rank test: Z = 8928351.5, p (Holm adjusted) < .001), and only then sharing (M = 0.30, SD = 0.69; answering vs. sharing Wilcoxon signed rank test: Z = 10317004.5, p (Holm adjusted) < .001). For self-directed hate, answering (M = 1.4, SD = 1.25) was more common than reporting (M = 0.93, SD = 1.31; answering vs. reporting, Wilcoxon signed rank test: Z = 3332546, p (Holm adjusted) < .001), followed by sharing (M = 0.79, SD = 1.13; reporting vs. sharing, Wilcoxon signed rank test: Z = 1678001, p (Holm adjusted) < .001). See also Figure 2.

Figure 3:

Distribution of answer choices for reactions to hate (by type of reaction and target of hate).

Note: Frequency of chosen answer options of answering, sharing, or reporting reactions when faced with self- or other-directed hateful comments. Error bars denote confidence intervals for proportions (Wald intervals).

Regarding the relationship between exposure to online hate and prosocial behavior, H3 predicted a non-linear (n-shaped) relationship between exposure to hate and prosocial reactions. Again, we built two models, one for self- and one for other-directed hate, and both showed significant coefficients at moderate to strong levels (according to Acock, 2014) in line with H3. Notably, the relationship between prosocial behavior and self-directed hate was larger than that between prosocial behavior and hate targeting others (see Table 1). Moreover, a graphical inspection of the model results shows that the relationship between exposure to hate and prosocial behavior is not exactly n-shaped. Rather, the positive relationship between exposure to hate and prosocial behavior flattens out towards higher levels of exposure (see Figure 4).

To investigate H4, testing the relationship between exposure to hate and negative emotional responses, we built linear models predicting each of the three negative emotional responses (anger, sadness, fear) separately. For each dependent variable, we built both simple models with only exposure to hate as a predictor variable and robustness check models that added several control variables. All models were in accordance with the hypothesized effect but with small effect sizes (e. g., a 1 % increase in exposure to other-directed hate increased participants’ self-reported anger by about 0.004 points on a 5-point Likert scale), showing that increased exposure to other-directed online hate is weakly associated with increases in negative emotional responses to encountering hate. Table 2 shows the main model results (see Appendix Table D.1 for models with covariates).

Moving to H5, positing a positive relationship between exposure to hate and motivations to withdraw from social media, we built two linear models predicting withdrawal reactions as a function of exposure to other-directed and self-directed hate respectively (see Table 3). As with H4, we created both models without and with control variables (key demographic variables, personality and political attitudes) as robustness checks.

Figure 4:

Predicted values of overall prosocial behavior.

Note: Predicted values of overall prosocial behavior as a function of exposure to hate. Plot A depicts this relationship for self-directed hate, plot B for other-directed hate. Corresponding model results are available in Table 3.

These robustness models also include an interaction term between encountered hate and prosocial behavior, as we initially preregistered a moderating effect of engaging prosocially with hate on withdrawal (see Appendix table D.4). The results of the models were mixed. Exposure to other-directed hate led to stronger withdrawal reactions (b = 0.16, 95 % CI [0.13, 0.18], p < .001) as predicted in H5, meaning that a 1 % increase in exposure to other-directed hate increased self-reported withdrawal intentions by about 0.002 points. Exposure to self-directed hate showed the opposite relationship (b = -0.15, 95 % CI [-0.23, -0.06], p < .001). The former result was robust to adding covariates or modeling the relationship with non-transformed exposure values, but the latter relationship did not hold when control variables were added (see Appendix table D.4 for full model results of robustness checks). In general, the strength of the relationship between frequency of exposure to hate and withdrawal motivations was weak, with models only explaining 0.2 % (for self-directed hate) to 0.6 % (for other-directed hate) of the observed variance in withdrawal motivations.

Overcoming free-riding by incentivizing bystander reactions

Looking at crowd moderation as an undersupplied public good makes it clear that to obtain efficient crowd moderation, we need to remedy the inbuilt incentives to freeride on the crowd moderation efforts of others. We can do so by altering the perceived individual costs or benefits, i. e., by altering how cumbersome it is to react to online hate (individual costs), or how much the individual stands to gain personally from reacting. This can be done in several ways, yet, we limit our discussion to two: (1) encouraging crowd moderation via platform design; and (2) encouraging crowd moderation via interventions.

Studies have shown promising results with regard to boosting bystander reactions by tweaking platform affordances. For example, increasing the visibility of audience size on platforms may reduce perceptions of costs of engaging in crowd moderation by providing “security in numbers” (Buerger, 2021), and may increase benefits by signaling the virtue of moderation to onlookers (DiFranzo et al., 2018; Taylor et al., 2019).

Other platform alterations have been tried to raise the individual’s perceived benefit of reacting to online hate. YouTube’s discontinued Heroes program (Perez, 2016) allowed users, civil society organizations and government agencies that correctly reported content in violation of YouTube Community Guidelines to report multiple videos simultaneously. As compensation for their free labor, “heroes” were rewarded with points that enabled them to access “sneak peeks at new products or even test them out” (Perez, 2016). An alternative reward structure could highlight users who regularly engage in crowd moderation. Facebook’s “Top Fan” badges and Community Awards that allow administrators to single out positive contributions by awarding members with badges like “Insightful” and “Conversation Starter,” could serve as inspiration (Hutchinson, 2019, 2021).

The second strategy of altering perceived costs and benefits of engaging in crowd moderation of online hate entails providing information and training competences. Traditional bystander programs focused on activating bystanders in an offline context in response to, e. g., sexual harassment, have proven effective when delivered as face-to-face training (Kettrey and Marx, 2019; Miller, 2014) and as online training (Ebers and Thomsen, 2022; Salazar et al., 2019), and some studies of interventions targeting online bystanders suggest similar positive results. For instance, findings by Naab et al. indicate that the experienced costs of reporting hate can be lowered by providing “obtrusive intervention information and emphasizing the need for user intervention in comments sections” (Naab et al., 2018, p. 783). Likewise, the cognitive load of contemplating whether something is hateful and necessitates intervention can be reduced by highlighting the “netiquette” of a site (Park et al., 2014). Building on general insights on behavior change interventions, it seems plausible that perceived costs of engaging in crowd moderation of online hate could be reduced by giving individuals advice on how to react, i. e., boost self-efficacy, and by showcasing the effectiveness of bystander intervention, i. e., boost response efficacy (Milne et al., 2000; Banyard et al., 2007; Michie et al., 2014). One opportunity would be to highlight the variety of possible online bystander reactions (reporting, rating, counter-speech, support of victim). Compared to offline bystanders, online bystanders have much greater choice in deciding to react in a visible or invisible way—options that carry different costs. Finally, perceived benefits could be increased by focusing intervention messages on “threat appraisal” (Milne et al. 2000), i. e., explaining how online hate constitutes a threat to democratic deliberation or showcasing how instating a norm of bystander engagement would benefit the individual personally in the future.

Future studies and the need for stronger tests

Our findings indicate that viewing a pleasant online debate environment produced through crowd moderation as an undersupplied public good constitutes a valuable future research agenda. Nevertheless, there are good reasons to advocate for more research of crowd moderation as a public good, especially in designs suited for causal identification of real-life behavior.

First, given that our findings rely on cross-sectional data and correlational analyses, a causal interpretation of our results is ill advised. Our findings need to be corroborated in stronger designs of an experimental nature. Experimental designs would be more suited to disentangle how a change in perceived costs or benefits of engaging in crowd moderation of online hate affects reactions. Whether in the form of survey experiments or classic public goods games, the aim should be of a testing nature.

Second, we rely on self-reported measures of key variables. We asked participants to indicate, e. g., how often they have been exposed to hate on social media and how they responded. Although we cannot rule out that respondents over-report prosocial responses to hate, we believe that the risk of overestimation is small since the electronic survey was anonymous and the average frequency of, e. g., reporting and counter-speech in our study is low (more than half report never having engaged in any kind of crowd moderation).

Third, surveying respondents in only one country limits the opportunities for generalization. Denmark has high levels of internet penetration and social media usage, and a relatively low degree of political polarization, which has been found to correlate with lower levels of online hate speech in political discussions (Bail, 2021). To learn whether our findings travel to countries with higher degrees of political polarization requires further studies. Nevertheless, our study is the most comprehensive examination in Europe of patterns in bystander reactions to online hate.

Lastly, while we have argued that interventions and platform design alterations can potentially mobilize more people to engage in crowd moderation, it remains to be seen whether such strategies are efficient at scale. The assumption that crowd moderation reduces the negative implications of online hate also needs further corroboration. Early indications show promise (Hangartner et al., 2021), but for it to be a public good, we must establish whether crowd moderation is rightly viewed as a good at all.