research-article

Open access

Make Reddit Great Again: Assessing Community Effects of Moderation Interventions on r/The_Donald

Authors:

Amaury Trujillo,

Stefano CresciAuthors Info & Claims

Proceedings of the ACM on Human-Computer Interaction, Volume 6, Issue CSCW2

Article No.: 526, Pages 1 - 28

https://doi.org/10.1145/3555639

Published: 11 November 2022 Publication History

Abstract

The subreddit r/The_Donald was repeatedly denounced as a toxic and misbehaving online community, reasons for which it faced a sequence of moderation interventions by Reddit administrators. It was quarantined in June 2019, restricted in February 2020, and finally banned in June 2020, but despite precursory work on the matter, the effects of this sequence of interventions are still unclear. In this work, we follow a multidimensional causal inference approach, with data containing more than 15M posts made in a time frame of 2 years, to examine the effects of such interventions inside and outside of the subreddit. We find that the interventions greatly reduced the activity of problematic users. However, the interventions also caused an increase in toxicity and led users to share more polarized and less factual news. In addition, the restriction had stronger effects than the quarantine, and core users of r/The_Donald suffered stronger effects than the rest of users. Overall, our results provide evidence that the interventions had mixed effects and paint a nuanced picture of the consequences of community-level moderation strategies. We conclude by reflecting on the challenges of policing online platforms and on the implications for the design and deployment of moderation interventions.

References

[1]

Kimberley Allison and Kay Bussey. 2020. Communal quirks and circlejerks: A taxonomy of processes contributing to insularity in online communities. In The 14th International AAAI Conference on Web and Social Media (ICWSM'20).

[2]

Christopher A Bail, Lisa P Argyle, Taylor W Brown, John P Bumpus, Haohan Chen, MB Fallin Hunzaker, Jaemin Lee, Marcus Mann, Friedolin Merhout, and Alexander Volfovsky. 2018. Exposure to opposing views on social media can increase political polarization. Proceedings of the National Academy of Sciences, Vol. 115, 37 (2018), 9216--9221.

[3]

Jason Baumgartner, Savvas Zannettou, Brian Keegan, Megan Squire, and Jeremy Blackburn. 2020. The Pushshift Reddit dataset. In The 14th International AAAI Conference on Web and Social Media (ICWSM'20). 830--839.

[4]

Yochai Benkler, Robert Faris, and Hal Roberts. 2018. Network propaganda: Manipulation, disinformation, and radicalization in American politics. Oxford University Press.

[5]

James Lopez Bernal, Steven Cummins, and Antonio Gasparrini. 2017. Interrupted time series regression for the evaluation of public health interventions: a tutorial. International Journal of Epidemiology, Vol. 46, 1 (2017), 348--355.

[6]

Alexandre Bovet and Hernán A Makse. 2019. Influence of fake news in Twitter during the 2016 US presidential election. Nature communications, Vol. 10, 1 (2019), 1--14.

[7]

Kay H Brodersen, Fabian Gallusser, Jim Koehler, Nicolas Remy, and Steven L Scott. 2015. Inferring causal impact using Bayesian structural time-series models. The Annals of Applied Statistics, Vol. 9, 1 (2015), 247--274.

[8]

Caitlin Ring Carlson and Luc S Cousineau. 2020. Are You Sure You Want to View This Community? Exploring the Ethics of Reddit's Quarantine Practice. Journal of Media Ethics, Vol. 35, 4 (2020), 202--213.

[9]

Stevie Chancellor, Jessica Annette Pater, Trustin Clear, Eric Gilbert, and Munmun De Choudhury. 2016. #thyghgapp: Instagram content moderation and lexical variation in pro-eating disorder communities. In The 19th ACM Conference on Computer-supported Cooperative Work And Social Computing (CSCW'16). 1201--1213.

Digital Library

[10]

Eshwar Chandrasekharan, Shagun Jhaver, Amy Bruckman, and Eric Gilbert. 2020. Quarantined! Examining the effects of a community-wide moderation intervention on Reddit. arXiv:2009.11483 (2020).

[11]

Eshwar Chandrasekharan, Umashanthi Pavalanathan, Anirudh Srinivasan, Adam Glynn, Jacob Eisenstein, and Eric Gilbert. 2017. You can't stay here: The efficacy of Reddit's 2015 ban examined through hate speech. In The 20th ACM Conference On Computer-Supported Cooperative Work And Social Computing (CSCW'17). 1--22.

Digital Library

[12]

Justin Cheng, Cristian Danescu-Niculescu-Mizil, and Jure Leskovec. 2014. How community feedback shapes user behavior. In The 8th International AAAI Conference on Web and Social Media (ICWSM'14).

[13]

Simon Copland. 2020. Reddit quarantined: Can changing platform affordances reduce hateful material online? Internet Policy Review, Vol. 9, 4 (2020), 1--26.

[14]

Anna L Cox, Sandy JJ Gould, Marta E Cecchinato, Ioanna Iacovides, and Ian Renfree. 2016. Design frictions for mindful interactions: The case for microboundaries. In The 34th CHI Conference on Human Factors in Computing Systems (CHI'16). 1389--1397.

Digital Library

[15]

Stefano Cresci. 2020. A decade of social bot detection. Commun. ACM, Vol. 63, 10 (2020), 72--83.

Digital Library

[16]

Stefano Cresci, Amaury Trujillo, and Tiziano Fagni. 2022. Personalized interventions for online moderation. In The 33rd ACM Conference on Hypertext and Social Media (HT'22). ACM, 248--251.

Digital Library

[17]

Giovanni Da San Martino, Stefano Cresci, Alberto Barrón-Cede no, Seunghak Yu, Roberto Di Pietro, and Preslav Nakov. 2020. A survey on computational propaganda detection. In The 29th International Joint Conference on Artificial Intelligence (IJCAI'20). 4826--4832.

[18]

Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated hate speech detection and the problem of offensive language. In The 11th International AAAI Conference on Web and Social Media (ICWSM'17).

[19]

Hongbo Deng, Michael R Lyu, and Irwin King. 2009. A generalized Co-HITS algorithm and its application to bipartite graphs. In The 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD'09). 239--248.

Digital Library

[20]

Roberto Di Pietro, Maurantonio Caprolu, Simone Raponi, and Stefano Cresci. 2021. New Dimensions of Information Warfare. Advances in Information Security, Vol. 84. Springer. 268 pages.

[21]

Nicholas Dias, Gordon Pennycook, and David G Rand. 2020. Emphasizing publishers does not effectively reduce susceptibility to misinformation on social media. Harvard Kennedy School Misinformation Review, Vol. 1, 1 (2020).

[22]

Claudia Flores-Saviaga, Brian Keegan, and Saiph Savage. 2018. Mobilizing the Trump train: Understanding collective action in a political trolling community. In The 12th International AAAI Conference on Web and Social Media (ICWSM'18).

[23]

Paula Fortuna, Juan Soler-Company, and Leo Wanner. 2021. How well do hate speech, toxicity, abusive and offensive language classification models generalize across datasets? Information Processing & Management, Vol. 58, 3 (2021), 102524.

Digital Library

[24]

Antigoni Maria Founta, Constantinos Djouvas, Despoina Chatzakou, Ilias Leontiadis, Jeremy Blackburn, Gianluca Stringhini, Athena Vakali, Michael Sirivianos, and Nicolas Kourtellis. 2018. Large scale crowdsourcing and characterization of Twitter abusive behavior. In The 12th International AAAI Conference on Web and Social Media (ICWSM'18).

[25]

Tiana Gaudette, Ryan Scrivens, Garth Davies, and Richard Frank. 2020. Upvoting extremism: Collective identity formation and the extreme right on Reddit. New Media & Society (2020), 1461444820958123.

[26]

Tarleton Gillespie. 2018. Custodians of the Internet: Platforms, Content Moderation, and the Hidden Decisions That Shape Social Media. Yale University Press.

[27]

Christoph Hanck, Martin Arnold, Alexander Gerber, and Martin Schmelzer. 2019. Introduction to Econometrics with R. University of Duisburg-Essen (2019), 1--9.

[28]

Manoel Horta Ribeiro, Shagun Jhaver, Savvas Zannettou, Jeremy Blackburn, Gianluca Stringhini, Emiliano De Cristofaro, and Robert West. 2021. Do platform migrations compromise content moderation? Evidence from r/The_Donald and r/Incels. In The 24th ACM Conference On Computer-Supported Cooperative Work And Social Computing (CSCW'21). 1--24.

[29]

Shagun Jhaver, Christian Boylston, Diyi Yang, and Amy Bruckman. 2021. Evaluating the effectiveness of deplatforming as a moderation strategy on Twitter. In The 24th ACM Conference On Computer-Supported Cooperative Work And Social Computing (CSCW'21). 1--30.

Digital Library

[30]

Andreas Jungherr, Oliver Posegga, and Jisun An. 2021. Populist supporters on Reddit: A comparison of content and behavioral patterns within publics of supporters of Donald Trump and Hillary Clinton. Social Science Computer Review (2021), 0894439321996130.

[31]

Sara Kiesler, Robert Kraut, Paul Resnick, and Aniket Kittur. 2012. Regulating behavior in online communities. Building successful online communities: Evidence-based social design (2012), 125--178.

Digital Library

[32]

Shamanth Kumar, Reza Zafarani, and Huan Liu. 2011. Understanding user migration patterns in social media. In The 25th AAAI Conference on Artificial Intelligence (AAAI'11).

[33]

David MJ Lazer, Matthew A Baum, Yochai Benkler, Adam J Berinsky, Kelly M Greenhill, Filippo Menczer, Miriam J Metzger, Brendan Nyhan, Gordon Pennycook, David Rothschild, et al. 2018. The science of fake news. Science, Vol. 359, 6380 (2018), 1094--1096.

[34]

Joan Massachs, Corrado Monti, Gianmarco De Francisci Morales, and Francesco Bonchi. 2020. Roots of Trumpism: Homophily and social feedback in Donald Trump support on Reddit. In The 12th ACM Conference on Web Science (WebSci'20). 49--58.

Digital Library

[35]

Marcelo Mendoza, Maurizio Tesconi, and Stefano Cresci. 2020. Bots in social and interaction networks: Detection and impact estimation. ACM Transactions on Information Systems, Vol. 39, 1 (2020), 1--32.

Digital Library

[36]

Alexandros Mittos, Savvas Zannettou, Jeremy Blackburn, and Emiliano De Cristofaro. 2020. “And We Will Fight for Our Race!” A Measurement Study of Genetic Testing Conversations on Reddit and 4chan. In The 14th International AAAI Conference on Web and Social Media (ICWSM'20), Vol. 14. 452--463.

[37]

Edward Newell, David Jurgens, Haji Mohammad Saleem, Hardik Vala, Jad Sassine, Caitrin Armstrong, and Derek Ruths. 2016. User migration in online social networks: A case study on Reddit during a period of community unrest. In The 10th International AAAI Conference on Web and Social Media (ICWSM'16).

[38]

Leonardo Nizzoli, Serena Tardelli, Marco Avvenuti, Stefano Cresci, and Maurizio Tesconi. 2021. Coordinated behavior on social media in 2019 UK General Election. In The 15th International AAAI Conference on Web and Social Media (ICWSM'21). 443--454.

[39]

Pujan Paudel, Jeremy Blackburn, Emiliano De Cristofaro, Savvas Zannettou, and Gianluca Stringhini. 2021. Soros, Child Sacrifices, and 5G: Understanding the Spread of Conspiracy Theories on Web Communities. arXiv:2111.02187 (2021).

[40]

Gordon Pennycook, Adam Bear, Evan T Collins, and David G Rand. 2020. The implied truth effect: Attaching warnings to a subset of fake news headlines increases perceived accuracy of headlines without warnings. Management Science, Vol. 66, 11 (2020), 4944--4957.

Digital Library

[41]

Ashwin Rajadesingan, Paul Resnick, and Ceren Budak. 2020. Quick, community-specific learning: How distinctive toxicity norms are maintained in political subreddits. In The 14th International AAAI Conference on Web and Social Media (ICWSM'20), Vol. 14. 557--568.

[42]

Tyler T Reny and Benjamin J Newman. 2021. The opinion-mobilizing effect of social protest against police violence: Evidence from the 2020 George Floyd protests. American Political Science Review (2021), 1--9.

[43]

Bernhard Rieder and Yarden Skop. 2021. The fabrics of machine moderation: Studying the technical, normative, and organizational structure of Perspective API. Big Data & Society, Vol. 8, 2 (2021), 20539517211046181.

[44]

Diana Rieger, Anna Sophie Kümpel, Maximilian Wich, Toni Kiening, and Georg Groh. 2021. Assessing the extent and types of hate speech in fringe communities: A case study of alt-right communities on 8chan, 4chan, and Reddit. Social Media Society, Vol. 7, 4 (2021), 20563051211052906.

[45]

Haji Mohammad Saleem and Derek Ruths. 2018. The aftermath of disbanding an online hateful community. arXiv:1804.07354 (2018).

[46]

Andrea L Schaffer, Timothy A Dobbins, and Sallie-Anne Pearson. 2021. Interrupted time series analysis using autoregressive integrated moving average (ARIMA) models: a guide for evaluating large-scale health interventions. BMC Medical Research Methodology, Vol. 21, 1 (2021), 1--12.

[47]

Steven L Scott and Hal R Varian. 2014. Predicting the present with Bayesian structural time series. International Journal of Mathematical Modelling and Numerical Optimisation, Vol. 5, 1--2 (2014), 4--23.

[48]

Ryan P Shepherd. 2020. Gaming Reddit's algorithm: r/The_Donald, amplification, and the rhetoric of sorting. Computers and Composition, Vol. 56 (2020), 102572.

[49]

Ahmed Soliman, Jan Hafer, and Florian Lemmerich. 2019. A characterization of political communities on Reddit. In The 30th ACM Conference on Hypertext and Social Media (HT'19). 259--263.

Digital Library

[50]

Claire Wardle and Hossein Derakhshan. 2017. Information disorder: Toward an interdisciplinary framework for research and policy making. Council of Europe, Vol. 27 (2017).

[51]

William Webber, Alistair Moffat, and Justin Zobel. 2010. A similarity measure for indefinite rankings. ACM Transactions on Information Systems (TOIS), Vol. 28, 4 (2010), 1--38.

Digital Library

[52]

Galen Weld, Maria Glenski, and Tim Althoff. 2021. Political Bias and Factualness in News Sharing across more than 100,000 Online Communities. In The 15th International AAAI Conference on Web and Social Media (ICWSM'21).

[53]

Tim Weninger. 2014. An exploration of submissions and discussions in social news: Mining collective intelligence of Reddit. Social Network Analysis and Mining, Vol. 4, 1 (2014), 173.

[54]

Savvas Zannettou. 2021. “I Won the Election!”: An Empirical Analysis of Soft Moderation Interventions on Twitter. In The 15th International AAAI Conference on Web and Social Media (ICWSM'21). 865--876.

[55]

Savvas Zannettou, Tristan Caulfield, Emiliano De Cristofaro, Nicolas Kourtelris, Ilias Leontiadis, Michael Sirivianos, Gianluca Stringhini, and Jeremy Blackburn. 2017. The Web centipede: Understanding how Web communities influence each other through the lens of mainstream and alternative news sources. In The 2017 Internet Measurement Conference (IMC'17). 405--417.

Digital Library

[56]

Savvas Zannettou, Tristan Caulfield, William Setzer, Michael Sirivianos, Gianluca Stringhini, and Jeremy Blackburn. 2019. Who let the trolls out? Towards understanding state-sponsored trolls. In The 11th ACM Conference on Web Science (WebSci'19). 353--362. io

Digital Library

Cited By

Bulat BWang HFujimoto SFrey S(2024)The psychology of volunteer moderators: Tradeoffs between participation, belonging, and norms in online community governanceNew Media & Society10.1177/14614448241259028Online publication date: 29-Jul-2024
https://doi.org/10.1177/14614448241259028
Di Marco NBrunetti SCinelli MQuattrociocchi W(2024)Post-hoc Evaluation of Nodes Influence in Information Cascades: The Case of Coordinated AccountsACM Transactions on the Web10.1145/3700644Online publication date: 17-Oct-2024
https://dl.acm.org/doi/10.1145/3700644
Prinster GSmith CTan CKeegan B(2024)Community Archetypes: An Empirical Framework for Guiding Research Methodologies to Reflect User Experiences of Sense of Virtual Community on RedditProceedings of the ACM on Human-Computer Interaction10.1145/36373108:CSCW1(1-33)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637310
Show More Cited By

Index Terms

Make Reddit Great Again: Assessing Community Effects of Moderation Interventions on r/The_Donald
1. Human-centered computing
  1. Collaborative and social computing
    1. Empirical studies in collaborative and social computing
2. Information systems
  1. Information systems applications
    1. Collaborative and social computing systems and tools
      1. Social networking sites
  2. World Wide Web
    1. Web applications
      1. Social networks

Recommendations

Do Platform Migrations Compromise Content Moderation? Evidence from r/The_Donald and r/Incels
CSCW2

When toxic online communities on mainstream platforms face moderation measures, such as bans, they may migrate to other platforms with laxer policies or set up their own dedicated websites. Previous work suggests that within mainstream platforms, ...
Quarantined! Examining the Effects of a Community-Wide Moderation Intervention on Reddit
Should social media platforms override a community’s self-policing when it repeatedly break rules? What actions can they consider? In light of this debate, platforms have begun experimenting with softer alternatives to outright bans. We examine one such ...
One of Many: Assessing User-level Effects of Moderation Interventions on r/The_Donald
WebSci '23: Proceedings of the 15th ACM Web Science Conference 2023

Evaluating the effects of moderation interventions is a task of paramount importance, as it allows assessing the success of content moderation processes. So far, intervention effects have been almost solely evaluated at the aggregated platform or ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction

Proceedings of the ACM on Human-Computer Interaction Volume 6, Issue CSCW2

CSCW

November 2022

8205 pages

EISSN:2573-0142

DOI:10.1145/3571154

Editor:
Jeff Nichols
Google

Issue’s Table of Contents

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 November 2022

Published in PACMHCI Volume 6, Issue CSCW2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
900
Total Downloads

Downloads (Last 12 months)507
Downloads (Last 6 weeks)44

Reflects downloads up to 22 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Bulat BWang HFujimoto SFrey S(2024)The psychology of volunteer moderators: Tradeoffs between participation, belonging, and norms in online community governanceNew Media & Society10.1177/14614448241259028Online publication date: 29-Jul-2024
https://doi.org/10.1177/14614448241259028
Di Marco NBrunetti SCinelli MQuattrociocchi W(2024)Post-hoc Evaluation of Nodes Influence in Information Cascades: The Case of Coordinated AccountsACM Transactions on the Web10.1145/3700644Online publication date: 17-Oct-2024
https://dl.acm.org/doi/10.1145/3700644
Prinster GSmith CTan CKeegan B(2024)Community Archetypes: An Empirical Framework for Guiding Research Methodologies to Reflect User Experiences of Sense of Virtual Community on RedditProceedings of the ACM on Human-Computer Interaction10.1145/36373108:CSCW1(1-33)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637310
Jhaver SRathi HSaha K(2024)Bystanders of Online Moderation: Examining the Effects of Witnessing Post-Removal ExplanationsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642204(1-9)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642204
Cima LMannocci LAvvenuti MTesconi MCresci S(2024)Coordinated Behavior in Information Operations on TwitterIEEE Access10.1109/ACCESS.2024.339348212(61568-61585)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3393482
Dalelio CBoyles DHolody KWeinhold W(2024)From Parkland to Nashville: an analysis of knowledge and framing activities in Reddit discussions surrounding news coverage of guns in the United StatesCogent Social Sciences10.1080/23311886.2024.235672410:1Online publication date: 24-May-2024
https://doi.org/10.1080/23311886.2024.2356724
Tardelli SNizzoli LTesconi MConti MNakov PDa San Martino GCresci S(2024)Temporal dynamics of coordinated online behavior: Stability, archetypes, and influenceProceedings of the National Academy of Sciences10.1073/pnas.2307038121121:20Online publication date: 6-May-2024
https://doi.org/10.1073/pnas.2307038121
Higham PModirrousta-Galian ASeabrooke T(2024)Mean rating difference scores are poor measures of discernment: The role of response criteriaCurrent Opinion in Psychology10.1016/j.copsyc.2023.10178556(101785)Online publication date: Apr-2024
https://doi.org/10.1016/j.copsyc.2023.101785
Nasirian SNogara GGiordano S(2024)Not My Fault: Studying the Necessity of the User Classification & Employment of Fine-Level User-Based Moderation Interventions in Social NetworksComplex Networks & Their Applications XII10.1007/978-3-031-53503-1_29(354-365)Online publication date: 29-Feb-2024
https://doi.org/10.1007/978-3-031-53503-1_29
Bouleimen APagan NCresci SUrman AGiordano S(2024)Dynamics of Toxic Behavior in the Covid-19 Vaccination DebateComplex Networks & Their Applications XII10.1007/978-3-031-53503-1_26(316-327)Online publication date: 29-Feb-2024
https://doi.org/10.1007/978-3-031-53503-1_26
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents