research-article

Harmful Design in User-Generated Games and its Ethical and Governance Challenges: An Investigation of Design Co-Ideation of Game Creators on Roblox

Authors:

Sam Moradzadeh,

Yubo KouAuthors Info & Claims

Proceedings of the ACM on Human-Computer Interaction, Volume 8, Issue CHI PLAY

Article No.: 311, Pages 1 - 31

https://doi.org/10.1145/3677076

Published: 15 October 2024 Publication History

Abstract

An increasing number of game platforms, such as Roblox, enable game creators to develop user-generated games (UGGs). Yet, these platforms often come under scrutiny for hosting UGGs that contain harmful content, ranging from sexually explicit material to Nazi-themed roleplay. Limited attention has been paid to how harmful UGGs are ideated by game creators. To address this question, we studied an online Roblox creator community, where Roblox creators collectively engage in design ideation to brainstorm design ideas for UGGs. Through an inductive thematic analysis, we found three primary ways where Roblox creators' design ideation becomes risky, including how Roblox creators generate risky game design ideas, navigate through policy boundaries to develop these ideas, and share strategies of bypassing moderation. Based on our findings, we discuss ethical and governance challenges facing user-generated games. We propose design implications to support game creators in developing ethical game design ideas and safe game designs.

References

[1]

Lucy A. Sparrow, Martin Gibbs, and Michael Arnold. 2021. The ethics of multiplayer game design and community management: industry perspectives and challenges. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 2021. 1--13.

Digital Library

[2]

Zainab Agha, Karla Badillo-Urquiola, and Pamela J. Wisniewski. 2023. 'Strike at the Root': Co-designing Real-Time Social Media Interventions for Adolescent Online Risk Prevention. Proc. ACM Hum.-Comput. Interact. 7, CSCW1 (April 2023), 1--32. https://doi.org/10.1145/3579625

Digital Library

[3]

Harry Alston. 2021. 15 Most Popular Roblox Games. TheGamer. Retrieved February 17, 2024 from https://www.thegamer.com/roblox-most-popular-games-most-players/

[4]

Florian Arendt, Sebastian Scherr, and Daniel Romer. 2019. Effects of exposure to self-harm on social media: Evidence from a two-wave panel study among young adults. New Media & Society 21, 11--12 (November 2019), 2422--2442. https://doi.org/10.1177/1461444819850106

[5]

Patrick Armstrong. 2024. Palworld Lets You Enslave Humans, and It Gets Even Worse. The Nerd Stash. Retrieved February 21, 2024 from https://thenerdstash.com/palworld-lets-you-enslave-humans-and-it-gets-even-worse/

[6]

Heather T. Battles. 2010. Exploring Ethical and Methodological Issues in Internet-Based Research with Adolescents. International Journal of Qualitative Methods 9, 1 (March 2010), 27--39. https://doi.org/10.1177/160940691000900104

[7]

Susanne E. Baumgartner, Patti M. Valkenburg, and Jochen Peter. 2010. Unwanted online sexual solicitation and risky sexual online behavior across the lifespan. Journal of Applied Developmental Psychology 31, 6 (November 2010), 439--447. https://doi.org/10.1016/j.appdev.2010.07.005

[8]

Rebecca Y. Bayeck. 2022. Positionality: The Interplay of Space, Context and Identity. International Journal of Qualitative Methods 21, (April 2022), 16094069221114745. https://doi.org/10.1177/16094069221114745

[9]

Wolfgang Bialas and Lothar Fritze. 2014. Nazi Ideology and Ethics. Cambridge Scholars Publishing.

[10]

Candice Biernesser, Craig J. R. Sewall, David Brent, Todd Bear, Christina Mair, and Jeanette Trauth. 2020. Social media use and deliberate self-harm among youth: A systematized narrative review. Children and Youth Services Review 116, (September 2020), 105054. https://doi.org/10.1016/j.childyouth.2020.105054

[11]

Sarane S. Boocock and James S. Coleman. 1966. Games with simulated environments in learning. Sociology of education (1966), 215--236.

[12]

Glenn A. Bowen. 2008. Naturalistic inquiry and the saturation concept: a research note. Qualitative research 8, 1 (2008), 137--152.

[13]

John Bradbury. 2023. The most popular Roblox games in 2024. Android Police. Retrieved February 17, 2024 from https://www.androidpolice.com/most-popular-roblox-games/

[14]

Russell Brandom. 2021. Roblox is struggling to moderate re-creations of mass shootings. The Verge. Retrieved May 4, 2023 from https://www.theverge.com/2021/8/17/22628624/roblox-moderation-trust-and-safety-terrorist-content-christchurch

[15]

Dawn Beverley Branley and Judith Covey. 2017. Is exposure to online content depicting risky behavior related to viewers? own risky behavior offline? Computers in Human Behavior 75, (2017), 283--287.

[16]

Dawn Beverley Branley and Judith Covey. 2018. Risky behavior via social media: The role of reasoned and social reactive pathways. Computers in Human Behavior 78, (January 2018), 183--191. https://doi.org/10.1016/j.chb.2017.09.036

Digital Library

[17]

Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77--101.

[18]

Virginia Braun, Victoria Clarke, Nikki Hayfield, Louise Davey, and Elizabeth Jenkinson. 2022. Doing Reflexive Thematic Analysis. In Supporting Research in Counselling and Psychotherapy?: Qualitative, Quantitative, and Mixed Methods Research, Sofie Bager-Charleson and Alistair McBeath (eds.). Springer International Publishing, Cham, 19--38. https://doi.org/10.1007/978--3-031--13942-0_2

[19]

Bussinesswire. 2018. Roblox Emerges as a Top Online Entertainment Platform for Kids and Teens in 2017. Retrieved May 4, 2023 from https://www.businesswire.com/news/home/20180321005322/en/Roblox-Emerges-as-a-Top-Online-Entertainment-Platform-for-Kids-and-Teens-in-2017

[20]

Clement Chau. 2010. YouTube as a participatory culture. New directions for youth development 2010, 128 (2010), 65--74.

[21]

Maria Cipollone, Catherine C. Schifter, and Rick A. Moffat. 2014. Minecraft as a creative tool: A case study. International Journal of Game-Based Learning (IJGBL) 4, 2 (2014), 1--14.

Digital Library

[22]

Victoria Clarke and Virginia Braun. 2021. Thematic Analysis?: A Practical Guide. SAGE Publications Ltd, London. Retrieved from http://digital.casalini.it/9781526417305

[23]

James Clayton and Jasmin Dyer. 2022. Roblox: The children's game with a sex problem. BBC News. Retrieved May 8, 2023 from https://www.bbc.com/news/technology-60314572

[24]

Francesca Comunello and Simone Mulargia. 2015. User-Generated Video Gaming: Little Big Planet and Participatory Cultures in Italy. Games and Culture 10, 1 (January 2015), 57--80. https://doi.org/10.1177/1555412014557028

[25]

Roblox Corporation. Jailbreak [Update]. Roblox. Retrieved February 21, 2024 from https://www.roblox.com/games/606849621/Jailbreak-Update

[26]

Create & Learn Team. 2023. Roblox Programming: Everything You Need to Know. Kids? Coding Corner | Create & Learn. Retrieved May 4, 2023 from https://www.create-learn.us/blog/roblox-programming/

[27]

Cecilia D'Anastasio. How 'Roblox' Became a Playground for Virtual Fascists. Wired. Retrieved May 4, 2023 from https://www.wired.com/story/roblox-online-games-irl-fascism-roman-empire/

[28]

Ella Daniel and Maya Benish-Weisman. 2019. Value development during adolescence: Dimensions of change and stability. Journal of Personality 87, 3 (June 2019), 620--632. https://doi.org/10.1111/jopy.12420

[29]

Brian Dean. 2023. Roblox User and Growth Stats 2023. Backlinko. Retrieved May 4, 2023 from https://backlinko.com/roblox-users

[30]

Anna K. Döring, Ella Daniel, and Ariel Knafo-Noam. 2016. Introduction to the Special Section Value Development from Middle Childhood to Early Adulthood'New Insights from Longitudinal and Genetically Informed Research. Social Development 25, 3 (August 2016), 471--481. https://doi.org/10.1111/sode.12177

[31]

Haihan Duan, Yiwei Huang, Yifan Zhao, Zhen Huang, and Wei Cai. 2022. User-Generated Content and Editors in Video Games: Survey and Vision. In 2022 IEEE Conference on Games (CoG), August 2022. 536--543. https://doi.org/10.1109/CoG51982.2022.9893717

Digital Library

[32]

Elsa. Top 6 Inappropriate Roblox Games Parents Should Know [2023]. Retrieved May 7, 2023 from https://www.airdroid.com/parent-control/inappropriate-roblox-games/

[33]

Cyrus Farivar. 2019. Extremists creep into Roblox, an online game popular with children. NBC News. Retrieved May 4, 2023 from https://www.nbcnews.com/tech/tech-news/extremists-creep-roblox-online-game-popular-children-n1045056

[34]

Suely Ferreira Deslandes, Tiago Coutinho, Taiza Ramos de Souza Costa Ferreira, and Roberta Matassoli Duran Flach. 2021. Online challenges among children and adolescents: Self-inflicted harm and social media strategies. Salud Colectiva 16, (April 2021), e3264. https://doi.org/10.18294/sc.2020.3264

[35]

Casey Fiesler. 2019. Ethical Considerations for Research Involving (Speculative) Public Data. Proc. ACM Hum.-Comput. Interact. 3, GROUP (December 2019), 1--13. https://doi.org/10.1145/3370271

Digital Library

[36]

Mary Flanagan, Jonathan Belman, Helen Nissenbaum, and Jim Diamond. 2007. A Method For Discovering Values in Digital Games. In DiGRA Conference, 2007.Retrieved February 14, 2024 from http://www.digra.org/dl/db/07311.46300.pdf

[37]

Mary Flanagan, Daniel C. Howe, and Helen Nissenbaum. 2005. Values at play: Design tradeoffs in socially-oriented game design. In Proceedings of the SIGCHI conference on human factors in computing systems, 2005. 751--760.

Digital Library

[38]

Mary Flanagan and Helen Nissenbaum. 2007. A game design methodology to incorporate social activist themes. In Proceedings of the SIGCHI conference on Human factors in computing systems, 2007. 181--190.

Digital Library

[39]

Rosie Flewitt*. 2005. Conducting research with young children: some ethical considerations. Early Child Development and Care 175, 6 (August 2005), 553--565. https://doi.org/10.1080/03004430500131338

[40]

Michael Flood. 2009. The harms of pornography exposure among children and young people. Child Abuse Review 18, 6 (November 2009), 384--400. https://doi.org/10.1002/car.1092

[41]

Batya Friedman, Peter Kahn, and Alan Borning. 2002. Value sensitive design: Theory and methods. University of Washington technical report 2, (2002), 12.

[42]

Batya Friedman, Peter H. Kahn, Alan Borning, and Alina Huldtgren. 2013. Value sensitive design and information systems. Early engagement and new technologies: Opening up the laboratory (2013), 55--95.

[43]

Fleur Gabriel. 2014. Sexting, Selfies and Self-Harm: Young People, Social Media and the Performance of Self-Development. Media International Australia 151, 1 (May 2014), 104--112. https://doi.org/10.1177/1329878X1415100114

[44]

Bharath Ganesh and Jonathan Bright. 2020. Countering Extremists on Social Media: Challenges for Strategic Communication and Content Moderation. Policy & Internet 12, 1 (2020), 6--19. https://doi.org/10.1002/poi3.236

[45]

Robert Gorwa, Reuben Binns, and Christian Katzenbach. 2020. Algorithmic content moderation: Technical and political challenges in the automation of platform governance. Big Data & Society 7, 1 (January 2020), 2053951719897945. https://doi.org/10.1177/2053951719897945

[46]

Keiran Hardy. 2020. Navigating Radicalization Concepts: A Role for the Harm Principle. In Radicalization and Counter-Radicalization, Derek M.D. Silva and Mathieu Deflem (eds.). Emerald Publishing Limited, 21--35. https://doi.org/10.1108/S1521--613620200000025002

[47]

Nick Haslam, Brodie C. Dakin, Fabian Fabiano, Melanie J. McGrath, Joshua Rhee, Ekaterina Vylomova, Morgan Weaving, and Melissa A. Wheeler. 2020. Harm inflation: Making sense of concept creep. European Review of Social Psychology 31, 1 (January 2020), 254--286. https://doi.org/10.1080/10463283.2020.1796080

[48]

Noe Vargas Hernandez, Jami J. Shah, and Steven M. Smith. 2010. Understanding design ideation mechanisms through multilevel aligned empirical studies. Design Studies 31, 4 (July 2010), 382--410. https://doi.org/10.1016/j.destud.2010.04.001

[49]

Andrew Gary Darwin Holmes. 2020. Researcher Positionality--A Consideration of Its Influence and Place in Qualitative Research--A New Researcher Guide. Shanlax International Journal of Education 8, 4 (2020), 1--10.

[50]

Miranda AH Horvath, Llian Alys, Kristina Massey, Afroditi Pina, Mia Scally, and Joanna R. Adler. 2013. Basically. porn is everywhere: a rapid evidence assessment on the effects that access and exposure to pornography has on children and young people. (2013). Retrieved February 12, 2024 from http://eprints.mdx.ac.uk/10692

[51]

Chuck Huff, Deborah G. Johnson, and Keith W. Miller. 2004. Virtual Harms and Real Responsibility. In Social, Ethical and Policy Implications of Information Technology. IGI Global, 98--117. https://doi.org/10.4018/978--1--59140--168--1.ch006

[52]

Jialun Aaron Jiang. 2020. Identifying and Addressing Design and Policy Challenges in Online Content Moderation. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (CHI EA '20), April 25, 2020. Association for Computing Machinery, New York, NY, USA, 1--7. https://doi.org/10.1145/3334480.3375030

Digital Library

[53]

Ben Jonson. 2005. Design ideation: the conceptual sketch in the digital age. Design Studies 26, 6 (November 2005), 613--624. https://doi.org/10.1016/j.destud.2005.03.001

[54]

Matthew Kay, Cynthia Matuszek, and Sean A. Munson. 2015. Unequal Representation and Gender Stereotypes in Image Search Results for Occupations. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, April 18, 2015. ACM, Seoul Republic of Korea, 3819--3828. https://doi.org/10.1145/2702123.2702520

Digital Library

[55]

Sean Keach. 2018. Roblox kids? game is a haven for twisted Jihadi, Nazi and KKK racist roleplay. The Sun. Retrieved May 4, 2023 from https://www.thesun.co.uk/tech/6710158/roblox-game-racist-jihad-nazi-kkk-racism-twin-towers-children/

[56]

Changsu Kim, Ming-Hua Jin, Jongheon Kim, and Namchul Shin. 2012. User perception of the quality, value, and utility of user-generated content. Journal of Electronic Commerce Research 13, 4 (2012), 305.

[57]

Tae Wan Kim and Kevin Werbach. 2016. More than just a game: ethical issues in gamification. Ethics Inf Technol 18, 2 (June 2016), 157--173. https://doi.org/10.1007/s10676-016--9401--5

[58]

Alex Knapp. Young Developers Earned Over $30M On Roblox In 2017, The Gaming Site Kids Visit More Than YouTube. Forbes. Retrieved June 18, 2023 from https://www.forbes.com/sites/alexknapp/2018/03/21/young-developers-earned-over-30m-on-roblox-in-2017-the-gaming-site-kids-visit-more-than-youtube/

[59]

Yubo Kou and Xinning Gui. 2023. Harmful Design in the Metaverse and How to Mitigate it: A Case Study of User-Generated Virtual Worlds on Roblox. In Proceedings of the 2023 ACM Designing Interactive Systems Conference (DIS '23), July 10, 2023. Association for Computing Machinery, New York, NY, USA, 175--188.

Digital Library

[60]

Mark Kretzschmar and Mel Stanfill. 2019. Mods as lightning rods: A typology of video game mods, intellectual property, and social benefit/harm. Social & legal studies 28, 4 (2019), 517--536.

[61]

Annakaisa Kultima. 2010. The organic nature of game ideation: game ideas arise from solitude and mature by bouncing. In Proceedings of the International Academic Conference on the Future of Game Design and Technology, May 06, 2010. ACM, Vancouver British Columbia Canada, 33--39. https://doi.org/10.1145/1920778.1920784

Digital Library

[62]

Britny Kutuchief. 2022. What is Roblox? Everything You Need to Know About the Social Gaming Platform. Social Media Marketing & Management Dashboard. Retrieved May 4, 2023 from https://blog.hootsuite.com/what-is-roblox/

[63]

Mandy Lau. 2022. Content moderation as language policy: Connecting commercial content moderation policies, regulations, and language policy. Working papers in Applied Linguistics and Linguistics at York 2, (March 2022), 1--12. https://doi.org/10.25071/2564--2855.11

[64]

Megan SC Lim, Elise R. Carrotte, and Margaret E. Hellard. 2015. The impact of pornography on gender-based violence, sexual health and well-being: what do we know? J Epidemiol Community Health (2015). Retrieved February 12, 2024 from https://jech.bmj.com/content/early/2015/05/27/jech-2015--205453?versioned=true&casa_token=FIXaNT5--3UsAAAAA:m9OoQndoX_q58AJxjasjUsYtZ-KoJVmQJOcttmOMqJWA1-NuIlNy0c3529mWOKR6FwFi4vvYoo2B

[65]

Sonia Livingstone and Leslie Haddon. 2009. EU Kids Online. Zeitschrift Für Psychologie/Journal of Psychology 217, 4 (2009), 236.

[66]

Sonia Livingstone, Lucyna Kirwil, Cristina Ponte, and Elisabeth Staksrud. 2014. In their own words: What bothers children online? European Journal of Communication 29, 3 (June 2014), 271--288. https://doi.org/10.1177/0267323114521045

[67]

Sonia Livingstone and Mariya Stoilova. 2021. The 4Cs: Classifying online risk to children. (2021). Retrieved May 30, 2024 from https://www.ssoar.info/ssoar/bitstream/handle/document/71817/ssoar-2021-livingstone_et_al-The_4Cs_Classifying_Online_Risk.pdf

[68]

Silicon Digital Group Ltd. ' MeepCity ' | RoMonitor Stats. ' MeepCity ' | RoMonitor Stats. Retrieved February 17, 2024 from https://romonitorstats.com/game/370731277/

[69]

m88youngling. 2021. How Many People Play LittleBigPlanet? The Truth! LBP Union. Retrieved May 28, 2024 from https://www.lbpunion.com/littlebigplanet/how-many-people-play-littlebigplanet/

[70]

Stephann Makri, Tsui-Ling Hsueh, and Sara Jones. 2019. Ideation as an Intellectual Information Acquisition and Use Context: Investigating Game Designers? Information-Based Ideation Behavior. Journal of the Association for Information Science and Technology 70, 8 (2019), 775--787. https://doi.org/10.1002/asi.24169

Digital Library

[71]

Marie-Helen Maras. 2017. Social media platforms: Targeting the 'found space'of terrorists. Journal of Internet Law 21, 2 (2017), 3--9.

[72]

Marilisaraccoglobal. Mom horrified to see her 7-year-old's Roblox character 'gang-raped' in popular online game - National | Globalnews.ca. Global News. Retrieved March 16, 2023 from https://globalnews.ca/news/4316449/roblox-gang-rape-7-year-old-girl/

[73]

Adrienne L. Massanari. 2015. Participatory Culture, Community, and Play. Peter Lang US. https://doi.org/10.3726/978--1--4539--1501--1

[74]

Cecile Meier, Jose Saorín, Alejandro Bonnet de León, and Alberto Guerrero Cobos. 2020. Using the roblox video game engine for creating virtual tours and learning about the sculptural heritage. International Journal of Emerging Technologies in Learning (iJET) 15, 20 (2020), 268--280.

[75]

Juliette H. Walma van der Molen. 2004. Violence and suffering in television news: Toward a broader conception of harmful television content for children. Pediatrics 113, 6 (2004), 1771--1775.

[76]

Teresa K Naab and Annika Sehl. 2017. Studies of user-generated content: A systematic review. Journalism 18, 10 (November 2017), 1256--1273. https://doi.org/10.1177/1464884916673557

[77]

James Newman. 2019. 33. Minecraft: User-Generated Content. In 33. Minecraft: User-Generated Content. New York University Press, 277--284. https://doi.org/10.18574/nyu/9781479830404.003.0037

[78]

Jonathan A. Obar and Steve Wildman. 2015. Social media definition and the governance challenge: An introduction to the special issue. Telecommunications Policy 39, 9 (October 2015), 745--750. https://doi.org/10.1016/j.telpol.2015.07.014

Digital Library

[79]

Gwenn Schurgin O'Keeffe and Kathleen Clarke-Pearson. 2011. The impact of social media on children, adolescents, and families. Pediatrics 127, 4 (2011), 800--804.

[80]

Jahna Otterbacher, Jo Bates, and Paul Clough. 2017. Competent Men and Warm Women: Gender Stereotypes and Backlash in Image Search Results. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, May 02, 2017. ACM, Denver Colorado USA, 6620--6631. https://doi.org/10.1145/3025453.3025727

Digital Library

[81]

Mina Park, Yao Sun, and Margaret L. McLaughlin. 2017. Social Media Propagation of Content Promoting Risky Health Behavior. Cyberpsychology, Behavior, and Social Networking 20, 5 (May 2017), 278--285. https://doi.org/10.1089/cyber.2016.0698

[82]

Simon Parkin. 2022. The trouble with Roblox, the video game empire built on child labour. The Observer. Retrieved May 4, 2023 from https://www.theguardian.com/games/2022/jan/09/the-trouble-with-roblox-the-video-game-empire-built-on-child-labour

[83]

David Pimentel. 2012. Criminal child neglect and the free range kid: Is overprotective parenting the new standard of care. Utah L. Rev. (2012), 947.

[84]

Anthony T. Pinter, Pamela J. Wisniewski, Heng Xu, Mary Beth Rosson, and Jack M. Caroll. 2017. Adolescent online safety: Moving beyond formative evaluations to designing solutions for the future. In Proceedings of the 2017 Conference on Interaction Design and Children, 2017. 352--357.

Digital Library

[85]

Hannah Pitt, Samantha L. Thomas, Amy Bestman, Mike Daube, and Jeffrey Derevensky. 2017. Factors that influence children's gambling attitudes and consumption intentions: lessons for gambling harm prevention research, policies and advocacy strategies. Harm Reduct J 14, (February 2017), 11. https://doi.org/10.1186/s12954-017-0136--3

[86]

Hector Postigo. 2003. From Pong to Planet Quake: Post-Industrial Transitions from Leisure to Work. Information, Communication & Society 6, 4 (December 2003), 593--607. https://doi.org/10.1080/1369118032000163277

[87]

Hector Postigo. 2007. Of Mods and Modders: Chasing Down the Value of Fan-Based Digital Game Modifications. Games and Culture 2, 4 (October 2007), 300--313. https://doi.org/10.1177/1555412007307955

[88]

Joshua Wolens published. 2024. Palworld players discover you can kidnap and sell humans just like pals, although you will be warned it's not very polite. PC Gamer. Retrieved February 21, 2024 from https://www.pcgamer.com/palworld-players-discover-you-can-kidnap-and-sell-humans-just-like-pals-although-you-will-be-warned-its-not-very-polite/

[89]

Yolanda (Linda) Reid Chassiakos, Jenny Radesky, Dimitri Christakis, Megan A. Moreno, Corinn Cross, COUNCIL ON COMMUNICATIONS AND MEDIA, David Hill, Nusheen Ameenuddin, Jeffrey Hutchinson, Alanna Levine, Rhea Boyd, Robert Mendelson, and Wendy Sue Swanson. 2016. Children and Adolescents and Digital Media. Pediatrics 138, 5 (November 2016), e20162593. https://doi.org/10.1542/peds.2016--2593

[90]

Pedro Reisinho, Rui Raposo, and Nelson Zagalo. 2023. The ethical dilemma of modding digital games: A literature review of the creation and distribution of mods. Convergence: The International Journal of Research into New Media Technologies (December 2023), 13548565231223933. https://doi.org/10.1177/13548565231223933

[91]

Luc Rinaldi,. 2024. Kids spend countless hours developing games for Roblox. Most never earn a penny. National Post. Retrieved from https://nationalpost.com/news/roblox-kids-online-gaming-platform

[92]

Lynne D. Roberts. 2015. Ethical issues in conducting qualitative research in online communities. Qualitative Research in Psychology 12, 3 (2015), 314--325.

[93]

Antonius J. van Rooij, Max Birk, and Simone van der Hof. 2021. Behavioral design in video games. (December 2021). Retrieved February 21, 2024 from https://osf.io/x9vhs/

[94]

Joel Ross, Oliver Holmes, and Bill Tomlinson. 2012. " Playing with Genre: User-Generated Game Design in LittleBigPlanet 2. LUCI Report 2012 3, (2012).

[95]

Daniel Ruby. 2023. Roblox Statistics 2023 - (Users, Revenue & Trends). Demand Sage. Retrieved May 4, 2023 from https://www.demandsage.com/how-many-people-play-roblox/

[96]

Lauren Rygg. 2015. School Shooting Simulations: At What Point Does Preparation Become More Harmful than Helpful. Child. Legal Rts. J. 35, 3 (2015), 215--228.

[97]

Marcelo Luis Barbosa dos Santos. 2021. The 'so-called' UGC: an updated definition of user-generated content in the age of social media. Online Information Review 46, 1 (January 2021), 95--113. https://doi.org/10.1108/OIR-06--2020-0258

[98]

Morgan Klaus Scheuerman, Jialun Aaron Jiang, Casey Fiesler, and Jed R. Brubaker. 2021. A Framework of Severity for Harmful Content Online. Proc. ACM Hum.-Comput. Interact. 5, CSCW2 (October 2021), 1--33. https://doi.org/10.1145/3479512

Digital Library

[99]

Carol F Scott, Gabriela Marcu, Riana Elyse Anderson, Mark W Newman, and Sarita Schoenebeck. 2023. Trauma-Informed Social Media: Towards Solutions for Reducing and Healing Online Harm. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, April 19, 2023. ACM, Hamburg Germany, 1--20. https://doi.org/10.1145/3544548.3581512

Digital Library

[100]

Emer Scully. 2022. Children's game platform Roblox is infiltrated with sexual games. Mail Online. Retrieved March 16, 2023 from https://www.dailymail.co.uk/news/article-10514185/Childrens-online-game-platform-Roblox-infested-sexually-explicit-games.html

[101]

Chirag Shah and Emily M. Bender. 2024. Envisioning Information Access Systems: What Makes for Good Tools and a Healthy Web? ACM Trans. Web 18, 3 (August 2024), 1--24. https://doi.org/10.1145/3649468

Digital Library

[102]

Jacob Shamsian. A mother says her 7-year-old daughter's character was raped in the kid's game 'Roblox.' Insider. Retrieved May 4, 2023 from https://www.insider.com/roblox-girl-avatar-rape-toxic-trolling-community-2018--7

[103]

Rohit Shewale. 2024. 25 Minecraft Statistics For 2024 (Users, Servers, & More). DemandSage. Retrieved May 28, 2024 from https://www.demandsage.com/minecraft-statistics/

[104]

Garriy Shteynberg, Michele J. Gelfand, and Kibum Kim. 2009. Peering into the 'Magnum Mysterium' of Culture: The Explanatory Power of Descriptive Norms. Journal of Cross-Cultural Psychology 40, 1 (January 2009), 46--69. https://doi.org/10.1177/0022022108326196

[105]

Miguel Sicart. 2005. The Ethics of Computer Game Design. DiGRA Conference.

[106]

Hugo Simão, Alexandre Bernardino, Jodi Forlizzi, and Tiago Guerreiro. 2022. Value Sensitive HRI. In Workshop" Longitudinal Studies" of the 2022 ACM/IEEE International Conference on Human-Robot Interaction, 2022.Retrieved June 3, 2024 from https://techandpeople.github.io/downloads/2022_hri.pdf

[107]

Hugo Simão, David Gonçalves, Ana C. Pires, Lúcia Abreu, Alexandre Bernardino, Jodi Forlizzi, and Tiago Guerreiro. 2024. 'I Want to Send a Message to My Friend': Exploring the Shift of Agency to Older Adults in HRI. Int J of Soc Robotics (April 2024). https://doi.org/10.1007/s12369-024-01128-y

[108]

Vera Slavtcheva-Petkova, Victoria Jane Nash, and Monica Bulger. 2015. Evidence on the extent of harms experienced by children as a result of online risks: implications for policy and research. Information, Communication & Society 18, 1 (January 2015), 48--62. https://doi.org/10.1080/1369118X.2014.934387

[109]

David Smahel, Hana Machackova, Giovanna Mascheroni, Lenka Dedkova, Elisabeth Staksrud, Kjartan Ólafsson, Sonia Livingstone, and Uwe Hasebrink. 2020. EU Kids Online 2020: Survey results from 19 countries. (2020).

[110]

Rachel Stonehouse. 2019. Roblox: 'I thought he was playing an innocent game.' BBC News. Retrieved May 4, 2023 from https://www.bbc.com/news/technology-48450604

[111]

Dean Takahashi. 2019. Roblox's 10 biggest games of all time - each with more than a billion plays. VentureBeat. Retrieved February 17, 2024 from https://venturebeat.com/business/robloxs-10-biggest-games-of-all-time-each-with-more-than-a-billion-plays/

[112]

Sean Targett, Victoria Verlysdonk, Howard J. Hamilton, and Daryl Hepting. 2012. A Study of User Interface Modifications in World of Warcraft. Game Studies 12, 2 (2012), 1--38.

[113]

Sarah-Kristin Thiel and Peter Lyle. 2019. Malleable games-a literature review on communities of game modders. In Proceedings of the 9th International Conference on Communities & Technologies-Transforming Communities, 2019. 198--209.

Digital Library

[114]

Nick Tornow. 2023. Enabling Creation of Anything, Anywhere, by Anyone. Roblox Blog. Retrieved September 11, 2023 from http://https%253A%252F%252Fblog.roblox.com%252F2023%252F03%252Fenabling-creation-anything-anywhere-anyone%252F

[115]

F. Ted Tschang and Janusz Szczypula. 2006. Idea creation, constructivism and evolution as key characteristics in the videogame artifact design process. European management journal 24, 4 (2006), 270--287.

[116]

Alexander Tsesis. 2017. Social media accountability for terrorist propaganda. Fordham L. Rev. 86, (2017), 605.

[117]

Steven Umbrello and Angelo F. De Bellis. 2018. A value-sensitive design approach to intelligent agents. In Artificial intelligence safety and security. Chapman and Hall/CRC, 395--409. Retrieved June 3, 2024 from https://www.taylorfrancis.com/chapters/edit/10.1201/9781351251389--26/value-sensitive-design-approach-intelligent-agents-steven-umbrello-angelo-de-bellis

[118]

Aimee Van Wynsberghe. 2020. Designing robots for care: Care centered value-sensitive design. In Machine ethics and robot ethics. Routledge, 185--211. Retrieved June 3, 2024 from https://www.taylorfrancis.com/chapters/edit/10.4324/9781003074991--17/designing-robots-care-care-centered-value-sensitive-design-aimee-van-wynsberghe

[119]

Ashley Marie Walker and Michael A. DeVito. 2020. ''More gay' fits in better?: Intracommunity Power Dynamics and Harms in Online LGBTQ+ Spaces. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, April 21, 2020. ACM, Honolulu HI USA, 1--15. https://doi.org/10.1145/3313831.3376497

Digital Library

[120]

Ryan Wallace. 2014. Modding: Amateur authorship and how the video game industry is actually getting it right. BYu L. REv. (2014), 219.

[121]

Pamela Wisniewski, Heng Xu, Mary Beth Rosson, Daniel F. Perkins, and John M. Carroll. 2016. Dear Diary: Teens Reflect on Their Weekly Online Risk Experiences. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, May 07, 2016. ACM, San Jose California USA, 3919--3930. https://doi.org/10.1145/2858036.2858317

Digital Library

[122]

Austin Wyatt and Jai Galliott. 2021. An empirical examination of the impact of cross-cultural perspectives on value sensitive design for autonomous systems. Information 12, 12 (2021), 527.

[123]

Leon Y. Xiao and Laura L. Henderson. 2021. Towards an Ethical Game Design Solution to Loot Boxes: a Commentary on King and Delfabbro. Int J Ment Health Addiction 19, 1 (February 2021), 177--192. https://doi.org/10.1007/s11469-019-00164--4

[124]

Greyson K. Young. 2022. How much is too much: the difficulties of social media content moderation. Information & Communications Technology Law 31, 1 (January 2022), 1--16. https://doi.org/10.1080/13600834.2021.1905593

[125]

José P. Zagal, Staffan Björk, and Chris Lewis. 2013. Dark patterns in the design of games. In Foundations of Digital Games 2013, 2013. .

[126]

2018. Roblox 'gang rape' shocks mother. BBC News. Retrieved March 16, 2023 from https://www.bbc.com/news/technology-44697788

[127]

Welcome to the Minecraft Official Site. Minecraft.net. Retrieved September 12, 2023 from https://www.minecraft.net/en-us

[128]

Platform Overview | Documentation - Roblox Creator Hub. Retrieved June 23, 2023 from https://create.roblox.com/docs

[129]

Room Moderation Guide. Rec Room. Retrieved September 5, 2023 from https://recroom.com/room-moderation-guide

[130]

Home - Roblox. Retrieved February 21, 2024 from https://www.roblox.com/home

[131]

Most popular Roblox games in 2022. Retrieved February 17, 2024 from https://superbiz.gg/blog/most-popular-roblox-games-in-2022

[132]

Luau - Roblox Documentation. Roblox. Retrieved May 4, 2023 from https://create.roblox.com/docs

[133]

Developer Forum | Roblox. Developer Forum | Roblox. Retrieved September 12, 2023 from https://devforum.roblox.com/

[134]

Roblox Gamedev - game development using roblox. Reddit. Retrieved September 12, 2023 from https://www.reddit.com/r/robloxgamedev/

[135]

Roblox Developers. Retrieved February 17, 2024 from https://www.reddit.com/r/RobloxDevelopers/

[136]

RobloxDev. Retrieved February 17, 2024 from https://www.reddit.com/r/RobloxDev/

[137]

Home. Ethical Games. Retrieved February 21, 2024 from http://ethicalgames.org/

[138]

Data API Terms - Reddit. Retrieved August 7, 2023 from https://www.redditinc.com/policies/data-api-terms

[139]

User Agreement - April 18, 2023 - Reddit. Retrieved May 28, 2024 from https://www.redditinc.com/policies/user-agreement-april-18--2023

[140]

Chernobyl | Chernobyl Accident | Chernobyl Disaster - World Nuclear Association. Retrieved February 12, 2024 from https://world-nuclear.org/information-library/safety-and-security/safety-of-plants/chernobyl-accident.aspx

[141]

9/11: As Events Unfold | Transportation Security Administration. Retrieved February 12, 2024 from https://www.tsa.gov/videos/911-events-unfold-0

[142]

There's a new initiative to create a game industry code of ethics. Retrieved February 21, 2024 from https://www.gamedeveloper.com/game-platforms/there-s-a-new-initiative-to-create-a-game-industry-code-of-ethics

[143]

Roblox Community Standards. Roblox Support. Retrieved August 8, 2023 from https://en.help.roblox.com/hc/en-us/articles/203313410-Roblox-Community-Standards

[144]

Facebook Community Standards | Transparency Center. Retrieved September 7, 2023 from https://transparency.fb.com/policies/community-standards/

[145]

YouTube Community Guidelines & Policies - How YouTube Works. YouTube Community Guidelines & Policies - How YouTube Works. Retrieved September 7, 2023 from https://www.youtube.com/howyoutubeworks/policies/community-guidelines/

[146]

Frequently Asked Questions | Documentation - Roblox Creator Hub. Retrieved May 28, 2024 from https://create.roblox.com/docs

Cited By

Zhang HAlhanahnah MAhmed FFatih DLeitner PAli-Eldin A(2024)Machine Learning Systems are Bloated and VulnerableACM SIGMETRICS Performance Evaluation Review10.1145/3673660.365506452:1(37-38)Online publication date: 13-Jun-2024
https://dl.acm.org/doi/10.1145/3673660.3655064
Zhang QLan YGuo KWang D(2024)Lipwatch: Enabling Silent Speech Recognition on Smartwatches using Acoustic SensingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596148:2(1-29)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659614
Zhang QLiu KWang D(2024)Sensing to Hear through MemoryProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36595988:2(1-31)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659598
Show More Cited By

Index Terms

Harmful Design in User-Generated Games and its Ethical and Governance Challenges: An Investigation of Design Co-Ideation of Game Creators on Roblox
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI

Recommendations

Designing game idea generation games
Future Play '08: Proceedings of the 2008 Conference on Future Play: Research, Play, Share

This paper introduces idea generation games designed for the use of game designers. Three games designed especially for generating new game ideas were developed in the GameSpace project that studies methods for design and evaluation of casual mobile ...
Creativity techniques in game design
Future Play '07: Proceedings of the 2007 conference on Future Play

Innovation and novelty are seen as important elements in game design but systematic tools and methods for producing creative ideas may be little known or poorly available, and creativity itself can be seen as something mystical [2] that cannot be ...
User centered game design: evaluating massive multiplayer online role playing games for second language acquisition
Sandbox '08: Proceedings of the 2008 ACM SIGGRAPH symposium on Video games

Unlike recreational games, serious games do more than entertain the player. Serious games promote acquisition of information and skills that are valued in both the virtual world and the real world. The challenge is to design and develop serious games ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction

Proceedings of the ACM on Human-Computer Interaction Volume 8, Issue CHI PLAY

CHI PLAY

October 2024

1726 pages

EISSN:2573-0142

DOI:10.1145/3700823

Editor:
Jeff Nichols
Apple Inc., United States

Issue’s Table of Contents

Copyright © 2024 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 October 2024

Published in PACMHCI Volume 8, Issue CHI PLAY

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Honorable Mention

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
263
Total Downloads

Downloads (Last 12 months)263
Downloads (Last 6 weeks)46

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang HAlhanahnah MAhmed FFatih DLeitner PAli-Eldin A(2024)Machine Learning Systems are Bloated and VulnerableACM SIGMETRICS Performance Evaluation Review10.1145/3673660.365506452:1(37-38)Online publication date: 13-Jun-2024
https://dl.acm.org/doi/10.1145/3673660.3655064
Zhang QLan YGuo KWang D(2024)Lipwatch: Enabling Silent Speech Recognition on Smartwatches using Acoustic SensingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596148:2(1-29)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659614
Zhang QLiu KWang D(2024)Sensing to Hear through MemoryProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36595988:2(1-31)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659598
Zhang HAlhanahnah MAhmed FFatih DLeitner PAli-Eldin AGaretto MMarin ACiucu FFanti GRighter R(2024)Machine Learning Systems are Bloated and VulnerableAbstracts of the 2024 ACM SIGMETRICS/IFIP PERFORMANCE Joint International Conference on Measurement and Modeling of Computer Systems10.1145/3652963.3655064(37-38)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3652963.3655064

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents