Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article
Public Access

Hate Raids on Twitch: Echoes of the Past, New Modalities, and Implications for Platform Governance

Published: 16 April 2023 Publication History

Abstract

In the summer of 2021, users on the livestreaming platform Twitch were targeted by a wave of "hate raids," a form of attack that overwhelms a streamer's chatroom with hateful messages, often through the use of bots and automation. Using a mixed-methods approach, we combine a quantitative measurement of attacks across the platform with interviews of streamers and third-party bot developers. We present evidence that confirms that some hate raids were highly-targeted, hate-driven attacks, but we also observe another mode of hate raid similar to networked harassment and specific forms of subcultural trolling. We show that the streamers who self-identify as LGBTQ+ and/or Black were disproportionately targeted and that hate raid messages were most commonly rooted in anti-Black racism and antisemitism. We also document how these attacks elicited rapid community responses in both bolstering reactive moderation and developing proactive mitigations for future attacks. We conclude by discussing how platforms can better prepare for attacks and protect at-risk communities while considering the division of labor between community moderators, tool-builders, and platforms.

References

[1]
Lindsay Blackwell, Jill Dimond, Sarita Schoenebeck, and Cliff Lampe. 2017. Classification and Its Consequences for Online Harassment: Design Insights from HeartMob. Proc. ACM Hum.-Comput. Interact., Vol. 1, CSCW, Article 24 (Dec. 2017), 19 pages. https://doi.org/10.1145/3134659
[2]
Jie Cai and Donghee Yvette Wohn. 2022. Coordination and Collaboration: How do Volunteer Moderators Work as a Team in Live Streaming Communities? Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (2022).
[3]
Carlos Castillo. 2016. Big crisis data: social media in disasters and time-critical situations. Cambridge University Press, Cambridge, UK.
[4]
Eshwar Chandrasekharan, Chaitrali Gandhi, Matthew Wortley Mustelier, and Eric Gilbert. 2019. Crossmod: A Cross-Community Learning-Based System to Assist Reddit Moderators. Proc. ACM Hum.-Comput. Interact., Vol. 3, CSCW, Article 174 (Nov. 2019), 30 pages. https://doi.org/10.1145/3359276
[5]
Justin Cheng, Michael Bernstein, Cristian Danescu-Niculescu-Mizil, and Jure Leskovec. 2017. Anyone Can Become a Troll: Causes of Trolling Behavior in Online Discussions. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (Portland, Oregon, USA) (CSCW '17). ACM, New York, NY, USA, 1217--1230. https://doi.org/10.1145/2998181.2998213
[6]
Marc Cheong and Vincent C S Lee. 2011. A microblogging-based approach to terrorism informatics: Exploration and chronicling civilian sentiment and response to terrorism events via Twitter. Information Systems Frontiers, Vol. 13, 1 (2011), 45--59. https://doi.org/10.1007/s10796-010--9273-x
[7]
Bill Cooney. [n.d.]. New Twitch stats reveal how few viewers are needed to be a ?top" streamer. Dexerto ( [n.,d.]). https://www.dexerto.com/entertainment/new-twitch-stats-reveal-how-few-viewers-are-needed-to-be-a-top-streamer-1527638/
[8]
Amanda C Cote. 2017. ?I Can Defend Myself": Women's Strategies for Coping With Harassment While Gaming Online. Games and Culture, Vol. 12, 2 (2017), 136--155. https://doi.org/10.1177/1555412015587603
[9]
John W Creswell. 2013. Qualitative Inquiry and Research Design: Choosing Among Five Traditions. SAGE, Thousand Oaks, CA.
[10]
Ralston Dacanay. [n.d.]. Twitch Streamer Creates Third-Party 'Panic Button' to Counter Hate Raids. DBLTAP ( [n.,d.]). https://www.dbltap.com/posts/twitch-streamer-creates-third-party-panic-button-to-counter-hate-raids-01fekqmgacvw
[11]
Cecilia D'Anastasio. [n.d.]. Twitch Sues Users Over Alleged ?Hate Raids' Against Streamers. Wired ( [n.,d.]). https://www.wired.com/story/twitch-sues-users-over-alleged-hate-raids/
[12]
Srayan Datta and Eytan Adar. 2019. Extracting inter-community conflicts in reddit. In Proceedings of the international AAAI conference on Web and Social Media.
[13]
Julian Dibbell. 1993. A Rape in Cyberspace: How an Evil Clown, a Haitian Trickster Spirit, Two Wizards, and a Cast of Dozens Turned a Database Into a Society. The Village Voice, Vol. December 23 (1993), 36--42. https://www.villagevoice.com/2005/10/18/a-rape-in-cyberspace/
[14]
Tuugrulcan Elmas, Rebekah Overdorf, Ahmed Furkan Özkalay, and Karl Aberer. 2022. Ephemeral Astroturfing Attacks: The Case of Fake Twitter Trends. In EuroS&P '22.
[15]
Jesse Fox and Wai Yen Tang. 2017. Women's experiences with general and sexual harassment in online video games: Rumination, organizational responsiveness, withdrawal, and coping strategies. New Media & Society, Vol. 19, 8 (2017), 1290--1307. https://doi.org/10.1177/1461444816635778
[16]
Sara Gordon. 1994. IRC and Security -- Can the two co-exist? (1994).
[17]
Kishonna L Gray. 2017. They're just too urban": Black gamers streaming on Twitch. In Digital Sociologies, Jessie Daniels, Karen Gregory, and Tressie McMillan Cottom (Eds.). Vol. 1. Policy Press, Bristol, England, Chapter 22, 355--368.
[18]
Nathan Grayson. [n.d.] a. Marginalized streamers beg Twitch to ?do better' in wake of hate raids, poor pay. The Washington Post ( [n.,d.]). https://www.washingtonpost.com/video-games/2021/08/11/twitch-do-better-hate-raids/
[19]
Nathan Grayson. [n.d.] b. Twitch hate raids are more than just a Twitch problem, and they're only getting worse. The Washington Post ( [n.,d.]). https://www.washingtonpost.com/video-games/2021/08/25/twitch-hate-raids-streamers-discord-cybersecurity/
[20]
Susan Herring, Kirk Job-Sluder, Rebecca Scheckler, and Sasha Barab. 2002. Searching for Safety Online: Managing ?Trolling" in a Feminist Forum. The Information Society, Vol. 18, 5 (2002), 371--384. https://doi.org/10.1080/01972240290108186
[21]
Dylan Horetski. [n.d.]. Twitch hate raids return in massive wave of attacks on LGBTQIA streamers. Dexerto ( [n.,d.]). https://www.dexerto.com/entertainment/twitch-hate-raids-return-in-massive-wave-of-attacks-on-lgbtqia-streamers-1781574/
[22]
Jialun Aaron Jiang, Charles Kiene, Skyler Middler, Jed R. Brubaker, and Casey Fiesler. 2019. Moderation Challenges in Voice-based Online Communities on Discord. Proc. ACM Hum.-Comput. Interact., Vol. 3, CSCW, Article 55 (Nov. 2019), 23 pages. https://doi.org/10.1145/3359157
[23]
Chris Kanich, Christian Kreibich, Kirill Levchenko, Brandon Enright, Geoffrey M Voelker, Vern Paxson, and Stefan Savage. 2008. Spamalytics: An empirical analysis of spam marketing conversion. In 15th ACM conference on Computer and communications security.
[24]
Charles Kiene, Andrés Monroy-Hernández, and Benjamin Mako Hill. 2016. Surviving an ?Eternal September": How an Online Community Managed a Surge of Newcomers. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI '16). ACM, New York, NY, USA, 1152--1156. https://doi.org/10.1145/2858036.2858356
[25]
Srijan Kumar, Justin Cheng, Jure Leskovec, and V.S. Subrahmanian. 2017. An Army of Me: Sockpuppets in Online Discussion Communities. In WWW '17.
[26]
Chen Ling, Utkucan Balci, Jeremy Blackburn, and Gianluca Stringhini. 2021. A First Look at Zoombombing. In IEEESP.
[27]
Richard MacKinnon. 1997. Virtual Rape. Journal of Computer-Mediated Communication, Vol. 2, 4 (1997), 1--2. https://doi.org/10.1111/j.1083--6101.1997.tb00200.x
[28]
Kaitlin Mahar, Amy X. Zhang, and David Karger. 2018. Squadbox: A Tool to Combat Email Harassment Using Friendsourced Moderation. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18). ACM, New York, NY, USA, Article 586, 13 pages. https://doi.org/10.1145/3173574.3174160
[29]
Enrico Mariconti, Guillermo Suarez-Tangil, Jeremy Blackburn, Emiliano De Cristofaro, Nicolas Kourtellis, Ilias Leontiadis, Jordi Luque Serrano, and Gianluca Stringhini. [n.d.]. "You Know What to Do" Proactive Detection of YouTube Videos Targeted by Coordinated Hate Attacks. Proceedings of the ACM on Human-Computer Interaction CSCW ( [n.,d.]).
[30]
Alice E Marwick. 2021. Morally Motivated Networked Harassment as Normative Reinforcement. Social Media Society, Vol. 7, 2 (2021), 20563051211021378. https://doi.org/10.1177/20563051211021378
[31]
Adrienne Massanari. 2017. #Gamergate and The Fappening: How Reddit's algorithm, governance, and culture support toxic technocultures. New Media & Society, Vol. 19, 3 (2017), 329--346.
[32]
Danaë Metaxa-Kakavouli, Paige Maas, and Daniel P. Aldrich. 2018. How Social Ties Influence Hurricane Evacuation Behavior. Proc. ACM Hum.-Comput. Interact., Vol. 2, CSCW, Article 122 (nov 2018), 16 pages. https://doi.org/10.1145/3274391
[33]
Andrés Monroy-Hernández, danah boyd, Emre Kiciman, Munmun De Choudhury, and Scott Counts. 2013. The New War Correspondents: The Rise of Civic Media Curation in Urban Warfare. In Proceedings of the 2013 Conference on Computer Supported Cooperative Work (San Antonio, Texas, USA) (CSCW '13). Association for Computing Machinery, New York, NY, USA, 1443--1452. https://doi.org/10.1145/2441776.2441938
[34]
David Moore, Colleen Shannon, Douglas J Brown, Geoffrey M Voelker, and Stefan Savage. 2006. Inferring internet denial-of-service activity. ACM Transactions on Computer Systems (TOCS) (2006).
[35]
Safiya Umoja Noble. 2018. Algorithms of Oppression. NYU Press, New York, NY, USA. https://doi.org/10.2307/j.ctt1pwt9w5
[36]
Leysia Palen and Kenneth M Anderson. 2016. Crisis informatics--New data for extraordinary times. Science, Vol. 353, 6296 (2016), 224--225. https://doi.org/10.1126/science.aag2579
[37]
Leysia Palen, Sarah Vieweg, Jeannette Sutton, Sophia B Liu, and Amanda Hughes. 2007. Crisis informatics: Studying crisis in a networked world. In Proceedings of the Third International Conference on E-Social Science. 7--9.
[38]
Manish Pandey. [n.d.]. Twitch announces new tools to fight hate raids. BBC ( [n.,d.]). https://www.bbc.com/news/newsbeat-58594732
[39]
Ash Parrish. [n.d.]. Twitch viewership noticeably dropped when streamers took a day off in protest. The Verge ( [n.,d.]). https://www.theverge.com/2021/9/2/22654534/streamers-twitch-walkout-viewership-drop
[40]
Whitney Phillips. 2011. LOLing at tragedy: Facebook trolls, memorial pages and resistance to grief online. First Monday, Vol. 16, 12 (2011), 12 pages. https://doi.org/10.5210/fm.v16i12.3168
[41]
Whitney Phillips. 2015. This is why we can't have nice things: Mapping the relationship between online trolling and mainstream culture. MIT Press, Cambridge, MA, USA.
[42]
Whitney Phillips. 2019. It Wasn't Just the Trolls: Early Internet Culture, ?Fun," and the Fires of Exclusionary Laughter. Social Media Society, Vol. 5, 3 (2019), 2056305119849493. https://doi.org/10.1177/2056305119849493
[43]
Blaine Polhamus. [n.d.]. Hate raids return to Twitch, another wave of attacks target LGBTQIA streamers. DOT ESPORTS ( [n.,d.]). https://dotesports.com/streaming/news/hate-raids-return-to-twitch-another-wave-of-attacks-target-lgbtqia-streamers
[44]
Sarah T. Roberts. 2016. Commercial Content Moderation: Digital Laborers' Dirty Work. In The Intersectional Internet: Race, Sex, Class and Culture Online, Safiya Umoja Noble and Brendesha M. Tynes (Eds.). Peter Lang Digital Formations series, New York, NY, USA, 147--160.
[45]
Morgan Klaus Scheuerman, Stacy M. Branham, and Foad Hamidi. 2018. Safe Spaces and Safe Places: Unpacking Technology-Mediated Experiences of Safety and Harm with Transgender People. Proc. ACM Hum.-Comput. Interact., Vol. 2, CSCW, Article 155 (Nov. 2018), 27 pages. https://doi.org/10.1145/3274424
[46]
Joseph Seering, Brianna Dym, Geoff Kaufman, and Michael Bernstein. 2022. Pride and Professionalization in Volunteer Moderation: Lessons for Effective in Platform-User Collaboration. Journal of Online Trust and Safety (2022).
[47]
Joseph Seering and Sanjay R. Kairam. 2022. Who Moderates on Twitch and What Do They Do? Quantifying Practices in Community Moderation on Twitch. Proc. ACM Hum.-Comput. Interact., Vol. 7, GROUP, Article 18 (dec 2022), 18 pages. https://doi.org/10.1145/3567568
[48]
Joseph Seering, Robert Kraut, and Laura Dabbish. 2017. Shaping Pro and Anti-Social Behavior on Twitch Through Moderation and Example-Setting. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (Portland, Oregon, USA) (CSCW '17). ACM, New York, NY, USA, 111--125. https://doi.org/10.1145/2998181.2998277
[49]
Joseph Seering, Tony Wang, Jina Yoon, and Geoff Kaufman. 2019. Moderator engagement and community development in the age of algorithms. New Media & Society, Vol. 21, 7 (2019), 1417--1443. https://doi.org/10.1177/1461444818821316
[50]
Bryan Semaan and Gloria Mark. 2011. Technology-Mediated Social Arrangements to Resolve Breakdowns in Infrastructure during Ongoing Disruption. ACM Trans. Comput.-Hum. Interact., Vol. 18, 4, Article 21 (12 2011), 21 pages. https://doi.org/10.1145/2063231.2063235
[51]
Anna DuVal Smith. 1999. Problems of Conflict Management in Virtual Communities. In Communities in Cyberspace 1st ed.), Marc A Smith and P Kollock (Eds.). Routledge, New York, NY, USA, 135--166.
[52]
Robert Soden and Leysia Palen. 2014. From Crowdsourced Mapping to Community Mapping: The Post-earthquake Work of OpenStreetMap Haiti. In COOP 2014 - Proceedings of the 11th International Conference on the Design of Cooperative Systems, 27--30 May 2014, Nice (France), Chiara Rossitto, Luigina Ciolfi, David Martin, and Bernard Conein (Eds.). Springer International Publishing, Cham, 311--326.
[53]
Robert Soden and Leysia Palen. 2016. Infrastructure in the Wild: What Mapping in Post-Earthquake Nepal Reveals about Infrastructural Emergence. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI '16). Association for Computing Machinery, New York, NY, USA, 2796--2807. https://doi.org/10.1145/2858036.2858545
[54]
Bijan Stephen. [n.d.]. The lockdown live-streaming numbers are out, and they're huge. The Verge ( [n.,d.]). https://www.theverge.com/2020/5/13/21257227/coronavirus-streamelements-arsenalgg-twitch-youtube-livestream-numbers
[55]
Hibby Thach, Samuel Mayworm, Daniel Delmonaco, and Oliver Haimson. 2022. (In)visible moderation: A digital ethnography of marginalized users and content moderation on Twitch and Reddit. New Media & Society (2022).
[56]
Kurt Thomas, Devdatta Akhawe, Michael Bailey, Dan Boneh, Sunny Consolvo, Nicola Dell, Zakir Durumeric, Patrick Gage Kelley, Deepak Kumar, Damon McCoy, Sarah Meiklejohn, Thomas Ristenpart, and Gianluca Stringhini. 2021. SoK: Hate, Harassment, and the Changing Landscape of Online Abuse. In IEEESP.
[57]
Kurt Thomas, Chris Grier, Dawn Song, and Vern Paxson. 2011. Suspended accounts in retrospect: an analysis of twitter spam. In ACM SIGCOMM conference on Internet measurement conference.
[58]
Kurt Thomas, Dmytro Iatskiv, Elie Bursztein, Tadek Pietraszek, Chris Grier, and Damon McCoy. 2014. Dialing back abuse on phone verified accounts. In ACM SIGSAC Conference on Computer and Communications Security.
[59]
Inc. Twitch Interactive. [n.d.]. Celebrate Yourself and Your Community with 350 New Tags. Twitch Blog ( [n.,d.]). https://blog.twitch.tv/en/2021/05/26/celebrate-yourself-and-your-community-with-350-new-tags/
[60]
Jessica Vitak, Kalyani Chadha, Linda Steiner, and Zahra Ashktorab. 2017. Identifying Women's Experiences With and Strategies for Mitigating Negative Effects of Online Harassment. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (Portland, Oregon, USA) (CSCW '17). ACM, New York, NY, USA, 1231--1245. https://doi.org/10.1145/2998181.2998337
[61]
Adam Yosilewitz. [n.d.]. StreamElements Analysis on Twitch Bullying. StreamElements ( [n.,d.]). https://blog.streamelements.com/streamelements-analysis-on-twitch-bullying-c3f2b2240318
[62]
Himanshu Zade, Kushal Shah, Vaibhavi Rangarajan, Priyanka Kshirsagar, Muhammad Imran, and Kate Starbird. 2018. From Situational Awareness to Actionability: Towards Improving the Utility of Social Media Data for Crisis Response. Proc. ACM Hum.-Comput. Interact., Vol. 2, CSCW, Article 195 (nov 2018), 18 pages. https://doi.org/10.1145/3274464

Cited By

View all
  • (2024)“You’ve Got to Put in the Time”: Neoliberal-Ableism and Disabled Streamers on TwitchSocieties10.3390/soc1406007514:6(75)Online publication date: 23-May-2024
  • (2024)PressProtect: Helping Journalists Navigate Social Media in the Face of Online HarassmentProceedings of the ACM on Human-Computer Interaction10.1145/36870488:CSCW2(1-34)Online publication date: 8-Nov-2024
  • (2024)Understanding and Mitigating New Harms in Immersive and Embodied Virtual Spaces: A Speculative Dystopian Design Fiction ApproachCompanion Publication of the 2024 Conference on Computer-Supported Cooperative Work and Social Computing10.1145/3678884.3681865(288-295)Online publication date: 11-Nov-2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction
Proceedings of the ACM on Human-Computer Interaction  Volume 7, Issue CSCW1
CSCW
April 2023
3836 pages
EISSN:2573-0142
DOI:10.1145/3593053
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 April 2023
Published in PACMHCI Volume 7, Issue CSCW1

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. community moderation
  2. online communities
  3. online harassment
  4. platform governance

Qualifiers

  • Research-article

Funding Sources

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)468
  • Downloads (Last 6 weeks)72
Reflects downloads up to 12 Nov 2024

Other Metrics

Citations

Cited By

View all
  • (2024)“You’ve Got to Put in the Time”: Neoliberal-Ableism and Disabled Streamers on TwitchSocieties10.3390/soc1406007514:6(75)Online publication date: 23-May-2024
  • (2024)PressProtect: Helping Journalists Navigate Social Media in the Face of Online HarassmentProceedings of the ACM on Human-Computer Interaction10.1145/36870488:CSCW2(1-34)Online publication date: 8-Nov-2024
  • (2024)Understanding and Mitigating New Harms in Immersive and Embodied Virtual Spaces: A Speculative Dystopian Design Fiction ApproachCompanion Publication of the 2024 Conference on Computer-Supported Cooperative Work and Social Computing10.1145/3678884.3681865(288-295)Online publication date: 11-Nov-2024
  • (2024)Exploring Gender and Racial/Ethnic Bias Against Video Game Streamers: Comparing Perceived Gameplay Skill and Viewer EngagementProceedings of the 19th International Conference on the Foundations of Digital Games10.1145/3649921.3650009(1-11)Online publication date: 21-May-2024
  • (2024)Governance Capture in a Self-Governing Community: A Qualitative Comparison of the Croatian, Serbian, Bosnian, and Serbo-Croatian WikipediasProceedings of the ACM on Human-Computer Interaction10.1145/36373388:CSCW1(1-26)Online publication date: 26-Apr-2024
  • (2024)Novel Approaches for Understanding and Mitigating Emerging New Harms in Immersive and Embodied Virtual Spaces: A Workshop at CHI 2024Extended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3636288(1-7)Online publication date: 11-May-2024
  • (2024)Third-Party Developers and Tool Development For Community Management on Live Streaming Platform TwitchProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642787(1-18)Online publication date: 11-May-2024
  • (2024)"I Got Flagged for Supposed Bullying, Even Though It Was in Response to Someone Harassing Me About My Disability.": A Study of Blind TikTokers’ Content Moderation ExperiencesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642148(1-15)Online publication date: 11-May-2024
  • (2024)“Vulnerable, Victimized, and Objectified”: Understanding Ableist Hate and Harassment Experienced by Disabled Content Creators on Social MediaProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641949(1-19)Online publication date: 11-May-2024
  • (2023)Networked Responses to Networked Harassment? Creators’ Coordinated Management of “Hate Raids” on TwitchSocial Media + Society10.1177/205630512311796969:2Online publication date: 6-Jun-2023

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Get Access

Login options

Full Access

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media