research-article

Public Access

Youth Trust in Social Media Companies and Expectations of Justice: Accountability and Repair After Online Harassment

Authors:

Sarita Schoenebeck,

Carol F. Scott,

Emma Grace Hurley,

Ellen SelkieAuthors Info & Claims

Proceedings of the ACM on Human-Computer Interaction, Volume 5, Issue CSCW1

Article No.: 2, Pages 1 - 18

https://doi.org/10.1145/3449076

Published: 22 April 2021 Publication History

Abstract

Social media platforms aspire to deliver fair resolutions after online harassment. Platforms rely on sanctions like removing content or banning users but these punitive responses provide little opportunity for justice or reparation for targets of harassment. This may be especially important for youth, who experience pervasive harassment which can have uniquely harmful effects on their wellbeing. We conducted a text-message based survey with 832 U.S. adolescents and young adults, ages 14-24, to explore their attitudes towards social media companies' responses to online harassment. We find that youth are twice as likely (41% versus 20%) not to trust social media companies' ability to achieve a fair resolution as they are to trust them. Nearly two-thirds (62%) of youth expressed a preference for an apology from the offender after online harassment, and they were twice as likely to prefer a private apology to a public one (29% versus 14%). Preferences also vary by identity, revealing how a one-size-fits-all approach can harm some youth while benefitting others. We reflect on the opportunities and risks associated with institutional trust and restorative justice for supporting youth who experience online harassment.

References

[1]

Asad, M. 2019. Prefigurative Design as a Method for Research Justice. Proceedings of the ACM on Human-Computer Interaction. 3, CSCW (Nov. 2019), 200:1--200:18.

Digital Library

[2]

Bailey, P.E., Slessor, G., Rieger, M., Rendell, P.G., Moustafa, A.A. and Ruffman, T. 2015. Trust and trustworthiness in young and older adults. Psychology and Aging. 30, 4 (2015), 977--986.

[3]

Barth, S. and de Jong, M.D.T. 2017. The privacy paradox -- Investigating discrepancies between expressed privacy concerns and actual online behavior -- A systematic literature review. Telematics and Informatics. 34, 7 (Nov. 2017), 1038--1058.

Digital Library

[4]

Battistella, E.L. 2014. Sorry about that: The Language of Public Apology. Oxford University Press.

[5]

Bradford, B., Grisel, F., Meares, T.L., Owens, E., Pineda, B.L., Shapiro, J., Tyler, T.R. and Peterman, D.E. 2019. Report Of The Facebook Data Transparency Advisory Group. Yale Justice Collaboratory.

[6]

Braithwaite, J. 1999. Restorative Justice: Assessing Optimistic and Pessimistic Accounts. Crime and Justice. 25, (Jan. 1999), 1--127.

[7]

Brestan, E.V. and Eyberg, S.M. Effective psychosocial treatments of conduct-disordered children and adolescents: 29 years, 82 studies, and 5,272 kids. Journal of Clinical Child Psychology. 27, 180--189.

[8]

Carstensen, L.L., Fung, H.H. and Charles, S.T. 2003. Socioemotional Selectivity Theory and the Regulation of Emotion in the Second Half of Life. Motivation and Emotion. 27, 2 (Jun. 2003), 103--123.

[9]

Chandrasekharan, E., Samory, M., Srinivasan, A. and Gilbert, E. 2017. The Bag of Communities: Identifying Abusive Behavior Online with Preexisting Internet Data. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, 2017), 3175--3187.

Digital Library

[10]

Choi, J.J., Bazemore, G. and Gilbert, M.J. 2012. Review of research on victims' experiences in restorative justice: Implications for youth justice. Children and Youth Services Review. 34, 1 (Jan. 2012), 35--42.

[11]

Citron, D.K. 2014. Hate Crimes in Cyberspace. Harvard University Press.

Digital Library

[12]

Cohen, N. 2019. Zuckerberg Wants Facebook to Build a Mind-Reading Machine. Wired.

[13]

Colquitt, J.A., Scott, B.A. and LePine, J.A. 2007. Trust, trustworthiness, and trust propensity: A meta-analytic test of their unique relationships with risk taking and job performance. Journal of Applied Psychology. 92, 4 (2007), 909--927.

[14]

Daly, K. and Tifft, L. 2007. The Limits of Restorative Justice. Handbook of Restorative Justice: A Global Perspective. Routledge.

[15]

DeJonckheere, M., Nichols, L.P., Moniz, M.H., Sonneville, K.R., Vydiswaran, V.V., Zhao, X., Guetterman, T.C. and Chang, T. 2017. MyVoice National Text Message Survey of Youth Aged 14 to 24 Years: Study Protocol. JMIR Research Protocols. 6, 12 (2017), e247.

[16]

Doherty, C., Kiley, J. and Inquiries, D. 20036USA202-419-4300 | M.-857-8562 | F.-419--4372 | M. Americans have become much less positive about tech companies' impact on the U.S. Pew Research Center.

[17]

Duggan, M. 2017. Online Harassment 2017. Pew Research Center: Internet, Science & Tech.

[18]

Facebook forms a special ethics team to prevent bias in its A.I. software: 2018. https://www.cnbc.com/2018/05/03/facebook-ethics-team-prevents-bias-in-ai-software.html. Accessed: 2020-08-20.

[19]

Facebook's algorithm bias only as neutral as their creators: http://www.washingtontimes.com/news/2016/may/16/facebooks-algorithm-bias-only-as-neutral-as-their-/. Accessed: 2016-09-19.

[20]

Friedman, B., Khan, P.H. and Howe, D.C. 2000. Trust online. Communications of the ACM. 43, 12 (Dec. 2000), 34--40.

Digital Library

[21]

Gillespie, T. 2018. Custodians of the Internet: Platforms, Content Moderation, and the Hidden Decisions That Shape Social Media. Yale University Press.

[22]

Goldberg, C. 2019. Nobody's Victim: Fighting Psychos, Stalkers, Pervs, and Trolls. Penguin.

[23]

Graycar, A. and Grabosky, P. 2002. The Cambridge Handbook of Australian Criminology. Cambridge University Press.

[24]

Greene, C., Sprott, J.B., Madon, N.S. and Jung, M. 2010. Punishing Processes in Youth Court: Procedural Justice, Court Atmosphere and Youths' Views of the Legitimacy of the Justice System1. Canadian Journal of Criminology and Criminal Justice. (Oct. 2010).

[25]

Growing to Trust: Evidence That Trust Increases and Sustains Well-Being Across the Life Span - Michael J. Poulin, Claudia M. Haase, 2015.

[26]

Guiso, L., Sapienza, P. and Zingales, L. 2004. The Role of Social Capital in Financial Development. American Economic Review. 94, 3 (Jun. 2004), 526--556.

[27]

Haimson, O.L. and Hoffmann, A.L. 2016. Constructing and enforcing "authentic" identity online: Facebook, real names, and non-normative identities. First Monday. 21, 6 (Jun. 2016).

[28]

Hand, C.A., Hankes, J. and House, T. 2012. Restorative justice: the indigenous justice system. Contemporary Justice Review. 15, 4 (Dec. 2012), 449--467.

[29]

Hardin, R. 2002. Trust and Trustworthiness. Russell Sage Foundation.

[30]

Hayes, H. 2006. Apologies and Accounts in Youth Justice Conferencing: Reinterpreting Research Outcomes. Contemporary Justice Review. 9, 4 (Dec. 2006), 369--385.

[31]

Hinds, L. 2007. Building Police-Youth Relationships: The Importance of Procedural Justice. Youth Justice. 7, 3 (Dec. 2007), 195--209.

[32]

Hoffmann, A.L. 2019. Where fairness fails: data, algorithms, and the limits of antidiscrimination discourse. Information, Communication & Society. 22, 7 (Jun. 2019), 900--915.

[33]

Hosseini, H., Kannan, S., Zhang, B. and Poovendran, R. 2017. Deceiving Google's Perspective API Built for Detecting Toxic Comments. arXiv:1702.08138 [cs]. (Feb. 2017).

[34]

Iyengar, V., Ghosh, D., Smith, T. and Krueger, and F. 2019. Age-Related Changes in Interpersonal Trust Behavior: Can Neuroscience Inform Public Policy? NAM Perspectives. (Jul. 2019).

[35]

Jhaver, S., Bruckman, A. and Gilbert, E. 2019. Does Transparency in Moderation Really Matter?: User Behavior After Content Removal Explanations on Reddit. Proc. ACM Hum.-Comput. Interact. CSCW, 2 (Nov. 2019).

[36]

Kaba, M. and Duda, J. 2018. Towards the horizon of abolition: A conversation with Mariame Kaba.

[37]

Kupchik, A. 2016. The Real School Safety Problem: The Long-Term Consequences of Harsh School Punishment. Univ of California Press.

[38]

Laloggia, J. and Inquiries 2019. U.S. public has little confidence in social media companies to determine offensive content. Pew Research Center.

[39]

Lang, F.R. and Carstensen, L.L. 2002. Time counts: Future time perspective, goals, and social relationships. Psychology and Aging. 17, 1 (2002), 125--139.

[40]

Levenson, J. 2017. Trauma-informed social work practice. Social Work. 62, 2 (2017), 105--113.

[41]

Madden, M. Teens Haven't Abandoned Facebook (Yet). Pew Research Center's Internet & American Life Project.

[42]

Mann, L., Harmoni, R. and Power, C. 1989. Adolescent decision-making: the development of competence. Journal of Adolescence. 12, 3 (Sep. 1989), 265--278.

[43]

McCaslin, W.D. 2013. Justice As Healing: Indigenous Ways. Living Justice Press.

[44]

McDonald, N., Schoenebeck, S. and Forte, A. 2019. Reliability and Inter-rater Reliability in Qualitative Research: Norms and Guidelines for CSCW and HCI Practice. Proceedings of ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '19) (2019).

[45]

Melton, A.P. 1995. Indigenous Justice Systems and Tribal Society Indian Tribal Courts and Justice: A Symposium. Judicature. 79, (1996 1995), 126--133.

[46]

Meyer, J.F. 1998. History Repeats itself: Restorative Justice in Native American Communities. Journal of Contemporary Criminal Justice. 14, 1 (Feb. 1998), 42--57.

[47]

Mingus, M. 2019. The Four Parts of Accountability: How To Give A Genuine Apology Part 1. Leaving Evidence.

[48]

Murphy, K. 2015. Does procedural justice matter to youth? Comparing adults' and youths' willingness to collaborate with police. Policing and Society. 25, 1 (Jan. 2015), 53--76.

[49]

Opinion | Could Restorative Justice Fix the Internet? - The New York Times: https://www.nytimes.com/2019/08/20/opinion/internet-harassment-restorative-justice.html. Accessed: 2021-01-06.

[50]

Penner, E.K., Viljoen, J.L., Douglas, K.S. and Roesch, R. 2014. Procedural justice versus risk factors for offending: Predicting recidivism in youth. Law and Human Behavior. 38, 3 (2014), 225--237.

[51]

Pharo, H., Sim, C., Graham, M., Gross, J. and Hayne, H. 2011. Risky business: Executive function, personality, and reckless behavior during adolescence and emerging adulthood. Behavioral Neuroscience. 125, 6 (2011), 970--978.

[52]

Phillips, W. 2015. This Is Why We Can't Have Nice Things: Mapping the Relationship Between Online Trolling and Mainstream Culture. MIT Press.

[53]

Prevent Cyberbullying: 2012. https://www.stopbullying.gov/cyberbullying/prevention/index.html. Accessed: 2019-08-23.

[54]

Rainie, L., Anderson, J. and Albright, J. 2017. The Future of Free Speech, Trolls, Anonymity and Fake News Online. Pew Research Center: Internet, Science & Tech.

[55]

Rainie, L., Keeter, S. and Perrin, A. 2019. Americans' Trust in Government, Each Other, Leaders. Pew Research Center - U.S. Politics & Policy.

[56]

Raja, S., Hasnain, M., Hoersch, M., Gove-Yin, S. and Rajagopalan, C. 2015. Trauma Informed Care in Medicine. Family & Community Health. 38, 3 (Jul. 2015), 216--226.

[57]

ReCal for Ordinal, Interval, and Ratio Data (OIR) - dfreelon.org: http://dfreelon.org/utils/recalfront/recal-oir. Accessed: 2020-05-18.

[58]

Regehr, C. and Gutheil, T. 2002. Apology, justice, and trauma recovery. Journal of the American Academy of Psychiatry and the Law. 30, 3 (2002), 425--430.

[59]

Richardson, A., Allen, J.A., Xiao, H. and Vallone, D. 2012. Effects of Race/Ethnicity and Socioeconomic Status on Health Information-Seeking, Confidence, and Trust. Journal of Health Care for the Poor and Underserved. 23, 4 (Oct. 2012), 1477--1493.

[60]

Roberts, S.T. 2019. Behind the Screen: Content Moderation in the Shadows of Social Media. Yale University Press.

[61]

Ronson, J. 2016. So You've Been Publicly Shamed. Penguin Publishing Group.

[62]

Rossner, M. 2013. Just emotions: rituals of restorative justice. Oxford University Press.

[63]

Hasinoff, A., Gibson, A.N., and Salehi, N. 2020. The promise of restorative justice in addressing online harm. Brookings.

[64]

Scheuerman, M.K., Branham, S.M. and Hamidi, F. 2018. Safe Spaces and Safe Places: Unpacking Technology-Mediated Experiences of Safety and Harm with Transgender People. Proc. ACM Hum.-Comput. Interact. 2, CSCW (Nov. 2018), 155:1--155:27.

Digital Library

[65]

Schoenebeck, S., Haimson, O. and Nakamura, L. 2020. Drawing from justice theories to support targets of online harassment. New Media & Society. (Mar. 2020), 1461444820913122.

[66]

Seering, J. 2020. Reconsidering Self-Moderation: the Role of Research in Supporting Community-Based Models for Online Content Moderation. Proceedings of the ACM on Human-Computer Interaction. 4, CSCW2 (Oct. 2020), 107:1--107:28.

Digital Library

[67]

Smith, A. 2018. How Americans View Tech Companies. Pew Research Center: Internet, Science & Tech.

[68]

Social Media Collective 2011. ?If you don't like it, don't use it. It's that simple." ORLY? Social Media Collective.

[69]

Sunshine, J. and Tyler, T.R. 2003. The Role of Procedural Justice and Legitimacy in Shaping Public Support for Policing. Law & Society Review. 37, 3 (2003), 513--548.

[70]

Suzor, N.P., West, S.M., Quodling, A. and York, J. 2019. What Do We Mean When We Talk About Transparency? Toward Meaningful Transparency in Commercial Content Moderation. International Journal of Communication. 13, 0 (Mar. 2019), 18.

[71]

This is how much Americans trust Facebook, Google, Apple, and other big tech companies: 2020. https://www.theverge.com/2020/3/2/21144680/verge-tech-survey-2020-trust-privacy-security-facebook-amazon-google-apple. Accessed: 2020-05-18.

[72]

Thomas, D.R. 2006. A General Inductive Approach for Analyzing Qualitative Evaluation Data. American Journal of Evaluation. 27, 2 (Jun. 2006), 237--246.

[73]

Tyler, T.R. 1988. What Is Procedural Justice - Criteria Used by Citizens to Assess the Fairness of Legal Procedures. Law & Society Review. 22, (1988), 103.

[74]

United States Population Growth by Region: https://www.census.gov/popclock/data_tables.php?component=growth. Accessed: 2020-05-19.

[75]

Upton-Clark, E. A Brief History of Pranks as Political Activism. Teen Vogue.

[76]

U.S. Census Bureau QuickFacts: United States: https://www.census.gov/quickfacts/fact/table/US/PST045219. Accessed: 2020-05-19.

[77]

Wenzel, M., Okimoto, T.G., Feather, N.T. and Platow, M.J. 2008. Retributive and Restorative Justice. Law and Human Behavior. 32, 5 (Oct. 2008), 375--389.

[78]

What is Trauma-Informed Care? http://socialwork.buffalo.edu/social-research/institutes-centers/institute-on-trauma-and-trauma-informed-care/what-is-trauma-informed-care.html. Accessed: 2020-05-21.

[79]

When It Comes to Abolition, Accountability Is a Gift: https://www.bitchmedia.org/article/mariame-kaba-josie-duffy-rice-rethinking-accountability-abolition. Accessed: 2021-01-06.

[80]

Wohn, D.Y. 2019. Volunteer Moderators in Twitch Micro Communities: How They Get Involved, the Roles They Play, and the Emotional Labor They Experience. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, 2019), 160:1-160:13.

Digital Library

[81]

Wulczyn, E., Thain, N. and Dixon, L. 2017. Ex Machina: Personal Attacks Seen at Scale. Proceedings of the 26th International Conference on World Wide Web (Republic and Canton of Geneva, Switzerland, 2017), 1391--1399.

Digital Library

[82]

Younger Americans less trusting of other people, institutions | Pew Research Center: https://www.pewresearch.org/fact-tank/2019/08/06/young-americans-are-less-trusting-of-other-people-and-key-institutions-than-their-elders/. Accessed: 2020-05-18.

Cited By

Bradford BKyprianides AAndrews WAston EClayton EO’Neill MWells H(2025)‘To whom am I speaking?’; Public responses to crime reporting via live chat with human versus AI police operatorsPolicing and Society10.1080/10439463.2025.2453437(1-17)Online publication date: 19-Jan-2025
https://doi.org/10.1080/10439463.2025.2453437
Swamy GAmes MSalehi N(2024)A playbook for mapping adolescent interactions with misinformation to perceptions of online harmHarvard Kennedy School Misinformation Review10.37016/mr-2020-161Online publication date: 30-Oct-2024
https://doi.org/10.37016/mr-2020-161
Seering JKhadka MHaghighi NYang TXi ZBernstein M(2024)Chillbot: Content Moderation in the BackchannelProceedings of the ACM on Human-Computer Interaction10.1145/36869418:CSCW2(1-26)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686941
Show More Cited By

Index Terms

Youth Trust in Social Media Companies and Expectations of Justice: Accountability and Repair After Online Harassment
1. Human-centered computing
  1. Collaborative and social computing
    1. Collaborative and social computing theory, concepts and paradigms
      1. Social media

Recommendations

A tool for help or harm? How associations between social networking use, social support, and mental health differ for sexual minority and heterosexual youth

Although use of social networking sites has been linked to both positive and negative changes in young people's mental health, it is likely that these contributions may vary based on users' motivations and social status. For sexual minority youth, for ...
Voluntary sexual exposure online among Swedish youth - social background, Internet behavior and psychosocial health

Studies have described the phenomenon of voluntary sexual exposure among youth online but only a few focus on the typical young person who has this experience. The purpose of this study was to investigate Swedish youth with experience of voluntary ...
Socio-Economic and Psychological Risks of Unemployed Youth in Developing Countries: Evidence From Hawassa City, Ethiopia

Urban youth unemployment has become a worldwide phenomenon in which Ethiopia is not an exception. Hawassa as among major cities of Ethiopia has become a home for many unemployed young people. This article has an overall objective of assessing the socio-...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction

Proceedings of the ACM on Human-Computer Interaction Volume 5, Issue CSCW1

CSCW

April 2021

5016 pages

EISSN:2573-0142

DOI:10.1145/3460939

Editor:
Jeff Nichols
Apple Inc., United States

Issue’s Table of Contents

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 April 2021

Published in PACMHCI Volume 5, Issue CSCW1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

26
Total Citations
View Citations
1,637
Total Downloads

Downloads (Last 12 months)554
Downloads (Last 6 weeks)70

Reflects downloads up to 13 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Bradford BKyprianides AAndrews WAston EClayton EO’Neill MWells H(2025)‘To whom am I speaking?’; Public responses to crime reporting via live chat with human versus AI police operatorsPolicing and Society10.1080/10439463.2025.2453437(1-17)Online publication date: 19-Jan-2025
https://doi.org/10.1080/10439463.2025.2453437
Swamy GAmes MSalehi N(2024)A playbook for mapping adolescent interactions with misinformation to perceptions of online harmHarvard Kennedy School Misinformation Review10.37016/mr-2020-161Online publication date: 30-Oct-2024
https://doi.org/10.37016/mr-2020-161
Seering JKhadka MHaghighi NYang TXi ZBernstein M(2024)Chillbot: Content Moderation in the BackchannelProceedings of the ACM on Human-Computer Interaction10.1145/36869418:CSCW2(1-26)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686941
Schulenberg KFreeman GLi LKnijnenburg B(2024)Does Who You Are or Appear to Be Matter?: Understanding Identity-Based Harassment in Social VR Through the Lens of (Mis)Perceived Identity RevelationProceedings of the ACM on Human-Computer Interaction10.1145/36869238:CSCW2(1-40)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686923
Qiwei LMcDonald AHaimson OSchoenebeck SGilbert E(2024)The Sociotechnical Stack: Opportunities for Social Computing Research in Non-Consensual Intimate MediaProceedings of the ACM on Human-Computer Interaction10.1145/36869148:CSCW2(1-21)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686914
Aliyu SKhan SMbodj AOsho OLi LKnijnenburg BCherubini M(2024)Participatory Design to Address Disclosure-Based CyberbullyingProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660716(1547-1565)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3660716
Urman AHannak AMakhortykh M(2024)User Attitudes to Content Moderation in Web SearchProceedings of the ACM on Human-Computer Interaction10.1145/36374238:CSCW1(1-27)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637423
Kim HLee JJang JKim J(2024)ReSPect: Enabling Active and Scalable Responses to Networked Online HarassmentProceedings of the ACM on Human-Computer Interaction10.1145/36373948:CSCW1(1-30)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637394
Cai JPatel ANaderi AWohn D(2024)Content Moderation Justice and Fairness on Social Media: Comparisons Across Different Contexts and PlatformsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650882(1-9)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650882
Batool ANaseem MToyama K(2024)Expanding Concepts of Non-Consensual Image-Disclosure Abuse: A Study of NCIDA in PakistanProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642871(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642871
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Figures

Tables

Media

View Issue’s Table of Contents