research-article

Enemy Within: Long-term Motivation Effects of Deep Player Behavior Models for Dynamic Difficulty Adjustment

Authors:

Jan David Smeddinck,

Rainer MalakaAuthors Info & Claims

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Pages 1 - 10

https://doi.org/10.1145/3313831.3376423

Published: 23 April 2020 Publication History

Abstract

Balancing games and producing content that remains interesting and challenging is a main cost factor in the design and maintenance of games. Dynamic difficulty adjustments (DDA) can successfully tune challenge levels to player abilities, but when implemented with classic heuristic parameter tuning (HPT) often turns out to be very noticeable, e.g. as "rubber-banding". Deep learning techniques can be employed for deep player behavior modeling (DPBM), enabling more complex adaptivity, but effects over frequent and longer-lasting game engagements, as well as how it compares to HPT has not been empirically investigated. We present a situated study of the effects of DDA via DPBM as compared to HPT on intrinsic motivation, perceived challenge and player motivation in a real-world MMORPG. The results indicate that DPBM can lead to significant improvements in intrinsic motivation and players prefer game experience episodes featuring DPBM over experience episodes with classic difficulty management.

Supplementary Material

MP4 File (paper296pv.mp4)

Preview video

Download
35.08 MB

MP4 File (pn4695vf.mp4)

Supplemental video

Download
100.03 MB

MP4 File (a296-pfau-presentation.mp4)

Download
223.30 MB

References

[1]

Nintendo Research Development 4. 1985. Super Mario Bros. Game [NES]. (13 September 1985). Nintendo, Kyoto, Japan.

[2]

Ernest Adams. 2002. Balancing Games with Positive Feedback. Gamasutra. com, January 4 (2002).

[3]

Dennis Ang and Alex Mitchell. 2017. Comparing Effects of Dynamic Difficulty Adjustment Systems on Video Game Experience. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play. ACM, 317--327.

Digital Library

[4]

Dennis Ang and Alex Mitchell. 2019. Representation and Frequency of Player Choice in Player-Oriented Dynamic Difficulty Adjustment Systems. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play. ACM, 589--600.

Digital Library

[5]

ArenaNet. 2005. Guild Wars. Game [PC]. (28 April 2005). ArenaNet, Bellevue, WA. Played 2019.

[6]

BioWare. 2012. Mass Effect 3. Game [PC,XBox360,PS3,WiiU]. (6 March 2012). BioWare, Edmonton, Kanada.

[7]

Capcom Production Studio 4. 2005. Resident Evil 4. Game [Gamecube]. (11 January 2005).

[8]

Thomas Constant and Guillaume Levieux. 2019. Dynamic Difficulty Adjustment Impact on Players' Confidence. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, 463.

Digital Library

[9]

Nevermind Creations. 2019. Korona:Nemesis. Game [PC]. (18 August 2019).

[10]

Mihaly Csikszentmihalyi. 2013. Flow: The psychology of happiness. Random House.

[11]

Thomas Debeauvais. 2016. Challenge and retention in games. Ph.D. Dissertation. UC Irvine.

[12]

Edward L Deci, Richard Koestner, and Richard M Ryan. 1999. A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychological bulletin 125, 6 (1999), 627.

[13]

Anders Drachen, Alessandro Canossa, and Georgios N Yannakakis. 2009. Player modeling using self-organization in Tomb Raider: Underworld. In 2009 IEEE symposium on computational intelligence and games. IEEE, 1--8.

[14]

Anders Drachen, Rafet Sifa, Christian Bauckhage, and Christian Thurau. 2012. Guns, swords and data: Clustering of player behavior in computer games in the wild. In 2012 IEEE conference on Computational Intelligence and Games (CIG). IEEE, 163--170.

[15]

Nintendo EAD. 1987. Zelda II: The Adventure of Link. Game [NES]. (14 January 1987). Nintendo EAD, Kyoto, Japan.

[16]

Nintendo EAD. 2014. Mario Kart 8. Game [WiiU,Switch]. (29 May 2014). Nintendo EAD, Kyoto, Japan. Played 2019.

[17]

William Rao Fernandes and Guillaume Levieux. 2019. ?-logit: Dynamic Difficulty Adjustment Using Few Data Points. In Joint International Conference on Entertainment Computing and Serious Games. Springer, 158--171.

Digital Library

[18]

Julian Frommel, Fabian Fischbach, Katja Rogers, and Michael Weber. 2018. Emotion-based Dynamic Difficulty Adjustment Using Parameterized Difficulty and Self-Reports of Emotion. In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play. ACM, 163--171.

Digital Library

[19]

Christoffer Holmgård, Antonios Liapis, Julian Togelius, and Georgios N Yannakakis. 2014a. Evolving personas for player decision modeling. In 2014 IEEE Conference on Computational Intelligence and Games. IEEE, 1--8.

[20]

Christoffer Holmgård, Antonios Liapis, Julian Togelius, and Georgios N Yannakakis. 2014b. Generative agents for player decision modeling in games. In FDG. Citeseer.

[21]

Christoffer Holmgård, Julian Togelius, and Georgios N Yannakakis. 2013. Decision making styles as deviation from rational action: A super mario case study. In Ninth Artificial Intelligence and Interactive Digital Entertainment Conference.

[22]

Dayana Hristova. 2017. Dynamic difficulty adjustment (DDA) in first person shooter (FPS) games. (2017).

[23]

Robin Hunicke. 2005. The case for dynamic difficulty adjustment in games. In Proceedings of the 2005 ACM SIGCHI International Conference on Advances in computer entertainment technology. ACM, 429--433.

Digital Library

[24]

Robin Hunicke and Vernell Chapman. 2004. AI for Dynamic Difficulty Adjustment in Games.

[25]

Changchun Liu, Pramila Agrawal, Nilanjan Sarkar, and Shuo Chen. 2009. Dynamic difficulty adjustment in computer games through real-time anxiety-based affective feedback. Int. Jrnl. of Human-Computer Interaction 25, 6 (2009), 506--529.

[26]

Ricardo Lopes, Ken Hilf, Luke Jayapalan, and Rafael Bidarra. 2013. Mobile adaptive procedural content generation. In Proceedings of the fourth workshop on Procedural Content Generation in Games (PCG 2013), Chania, Crete, Greece.

[27]

Tobias Mahlmann, Anders Drachen, Julian Togelius, Alessandro Canossa, and Georgios N Yannakakis. 2010. Predicting player behavior in tomb raider: Underworld. In Proceedings of the 2010 IEEE Conference on Computational Intelligence and Games. IEEE, 178--185.

[28]

Philipp Mayring. 2010. Qualitative inhaltsanalyse. In Handbuch qualitative Forschung in der Psychologie. Springer, 601--613.

[29]

Edward McAuley, Terry Duncan, and Vance V Tammen. 1989. Psychometric properties of the Intrinsic Motivation Inventory in a competitive sport setting: A confirmatory factor analysis. Research quarterly for exercise and sport 60, 1 (1989), 48--58.

[30]

Olana Missura and Thomas Gärtner. 2009. Player Modeling for Intelligent Difficulty Adjustment. In Discovery Science, João Gama, Vítor Santos Costa, Alípio Mário Jorge, and Pavel B. Brazdil (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 197--211.

[31]

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, and others. 2015. Human-level control through deep reinforcement learning. Nature 518, 7540 (2015), 529.

[32]

Fausto Mourato, Fernando Birra, and Manuel Próspero dos Santos. 2014. Difficulty in action based challenges: success prediction, players' strategies and profiling. In Proceedings of the 11th Conference on Advances in Computer Entertainment Technology. ACM, 9.

Digital Library

[33]

NCsoft. 2003. Lineage 2. Game [PC]. (1 October 2003). NCSoft, Seongnam, South Korea.

[34]

NCsoft. 2008. Aion. Game [PC]. (25 September 2008). NCSoft, Seongnam, South Korea. Played August 2019.

[35]

Pedro A Nogueira, Vasco Torres, Rui Rodrigues, Eugénio Oliveira, and Lennart E Nacke. 2016. Vanishing scares: biofeedback modulation of affective player experiences in a procedural horror game. Journal on Multimodal User Interfaces 10, 1 (2016), 31--62.

[36]

Juan Ortega, Noor Shaker, Julian Togelius, and Georgios N Yannakakis. 2013. Imitating human playing styles in super mario bros. Entertainment Computing 4, 2 (2013), 93--104.

[37]

Johannes Pfau, Jan David Smeddinck, Ioannis Bikas, and Rainer Malaka. 2020. Bot or not? User Perceptions of Player Substitution with Deep Player Behavior Models. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. ACM.

Digital Library

[38]

Johannes Pfau, Jan David Smeddinck, and Rainer Malaka. 2018. Towards Deep Player Behavior Models in MMORPGs. In Annual Symp. on Computer-Human Interaction in Play Ext. Abstracts (CHI PLAY '18). ACM, NY, NY, USA, 381--92.

[39]

Johannes Pfau, Jan David Smeddinck, and Rainer Malaka. 2019. Deep Player Behavior Models: Evaluating a Novel Take on Dynamic Difficulty Adjustment. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, LBW0171.

Digital Library

[40]

Mike Preuss, Thomas Pfeiffer, Vanessa Volz, and Nicolas Pflanzl. 2018. Integrated Balancing of an RTS Game: Case Study and Toolbox Refinement. In 2018 IEEE Conference on Computational Intelligence and Games (CIG). IEEE, 1--8.

Digital Library

[41]

Andrew K Przybylski, C Scott Rigby, and Richard M Ryan. 2010. A motivational model of video game engagement. Review of general psychology 14, 2 (2010), 154--166.

[42]

Nintendo RD1. 2002. Metroid Fusion. Game [GBA]. (18 November 2002). Nintendo RD1, Kyoto, Japan.

[43]

Andrew Rollings and Ernest Adams. 2003. Andrew Rollings and Ernest Adams on game design. New Riders.

[44]

Robert Rosenthal and Kermit L Fode. 1963. The effect of experimenter bias on the performance of the albino rat. Behavioral Science 8, 3 (1963), 183--189.

[45]

Richard M Ryan and Edward L Deci. 2000. Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary educational psychology 25, 1 (2000), 54--67.

[46]

Lingdao Sha, Souju He, Junping Wang, Jiajian Yang, Yuan Gao, Yidan Zhang, and Xinrui Yu. 2010. Creating appropriate challenge level game opponent by the use of dynamic difficulty adjustment. In 2010 Sixth International Conference on Natural Computation, Vol. 8. IEEE, 3897--3901.

[47]

Noor Shaker, Julian Togelius, and Georgios N Yannakakis. 2016. The experience-driven perspective. In Procedural Content Generation in Games. Springer, 181--194.

[48]

David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, and Laurent Sifre et al. 2016. Mastering the game of Go with deep neural networks and tree search. Nature 529 (2016), 484--489.

[49]

Adam M. Smith, Chris Lewis, Kenneth Hullet, Gillian Smith, and Anne Sullivan. 2011. An Inclusive View of Player Modeling. In Proceedings of the 6th International Conference on Foundations of Digital Games (FDG '11). ACM, NY, NY, USA, 301--303.

Digital Library

[50]

David Stammer, Tobias Günther, and Mike Preuss. 2015. Player-adaptive spelunky level generation. In 2015 IEEE Conference on Computational Intelligence and Games (CIG). IEEE, 130--137.

[51]

Alexander Streicher and Jan D. Smeddinck. 2016. Personalized and Adaptive Serious Games. In Entertainment Computing and Serious Games, Ralf Dörner, Stefan Göbel, Michael Kickmeier-Rust, Maic Masuch, and Katharina Zweig (Eds.). Lecture Notes in Computer Science, Vol. 9970. Springer International Publishing, Cham, 332--377.

[52]

Mirko Suznjevic and Maja Matijasevic. 2010. Why MMORPG players do what they do: relating motivations to action categories. International Journal of Advanced Media and Communication 4, 4 (2010), 405--424.

Digital Library

[53]

Marco Tamassia, William Raffe, Rafet Sifa, Anders Drachen, Fabio Zambetta, and Michael Hitchens. 2016. Predicting player churn in destiny: A hidden markov models approach to predicting player departure in a major online game. In 2016 IEEE Conference on Computational Intelligence and Games (CIG). IEEE, 1--8.

Digital Library

[54]

Gerald Tesauro. 1994. TD-Gammon, a self-teaching backgammon program, achieves master-level play. Neural computation 6, 2 (1994), 215--219.

[55]

Valve. 2008. Left 4 Dead. Game [PC]. (18 November 2008). Valve, Bellevue, WA, USA. Played 2017.

[56]

Hao Wang and Chuen-Tsai Sun. 2011. Game reward systems: Gaming experiences and social meanings. In DiGRA Conference, Vol. 114.

[57]

Su Xue, Meng Wu, John Kolen, Navid Aghdaie, and Kazi A Zaman. 2017. Dynamic difficulty adjustment for maximized engagement in digital games. In Proceedings of the 26th International Conference on World Wide Web Companion. International World Wide Web Conferences Steering Committee, 465--471.

Digital Library

[58]

Georgios N Yannakakis, Pieter Spronck, Daniele Loiacono, and Elisabeth André. 2013. Player modeling. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik.

Cited By

Kristensen JBurelli P(2024)Difficulty Modelling in Mobile Puzzle GamesInternational Journal of Computer Games Technology10.1155/2024/55923732024Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1155/2024/5592373
Pfau JSeif El-Nasr M(2024)On Video Game Balancing: Joining Player- and Data-Driven AnalyticsGames: Research and Practice10.1145/36758072:3(1-30)Online publication date: 30-Aug-2024
https://dl.acm.org/doi/10.1145/3675807
Pfau JCharan MKleinman ESeif El-Nasr M(2024)Damage Optimization in Video Games: A Player-Driven Co-Creative ApproachProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642643(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642643
Show More Cited By

Index Terms

Enemy Within: Long-term Motivation Effects of Deep Player Behavior Models for Dynamic Difficulty Adjustment

Recommendations

Deep Player Behavior Models: Evaluating a Novel Take on Dynamic Difficulty Adjustment
CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems

Finding and maintaining the right level of challenge with respect to the individual abilities of players has long been in the focus of game user research (GUR) and game development (GD). The right difficulty balance is usually considered a prerequisite ...
Diversifying dynamic difficulty adjustment agent by integrating player state models into Monte-Carlo tree search
Abstract
Game developers have employed dynamic difficulty adjustment (DDA) in designing game artificial intelligence (AI) to improve players’ game experience by adjusting the skill of game agents. Traditional DDA agents depend on player proficiency only ...
Highlights
- Adapting real-time video game difficulty with player’s affective states-based MCTS.
- DDA agent predicts the players’ state using occurred logs and simulated logs by MCTS.
- DDA agent adapts game difficulty based on predicted player’s ...
The effect of multiplayer dynamic difficulty adjustment on the player experience of video games
CHI EA '14: CHI '14 Extended Abstracts on Human Factors in Computing Systems

Multiplayer Dynamic Difficulty Adjustment (mDDA) is a method of reducing the difference in player performance and subsequent challenge in competitive multiplayer video games. As a balance of between player skill and challenge experienced is necessary ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

April 2020

10688 pages

ISBN:9781450367080

DOI:10.1145/3313831

General Chairs:
Regina Bernhaupt
Eindhoven University of Technology, Netherlands
,
Florian 'Floyd' Mueller
Monash University, Australia
,
David Verweij
Newcastle University, UK
,
Josh Andres
RMIT, Australia
,
Program Chairs:
Joanna McGrenere
University of British Columbia, Canada
,
Andy Cockburn
University of Canterbury, New Zealand
,
Ignacio Avellino
University of Maryland Baltimore County, USA
,
Alix Goguey
Grenoble Alpes University, France
,
Pernille Bjørn
University of Copenhagen, Denmark
,
Shengdong (Shen) Zhao
National University of Singapore, Singapore
,
Briane Paul Samson
Future University Hakodate, Japan & De La Salle University, Philippines
,
Rafal Kocielnik
University of Washington, USA

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 April 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

German Research Foundation (DFG)

Conference

CHI '20

Sponsor:

SIGCHI

CHI '20: CHI Conference on Human Factors in Computing Systems

April 25 - 30, 2020

HI, Honolulu, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI '25

Sponsor:
sigchi

CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

28
Total Citations
View Citations
769
Total Downloads

Downloads (Last 12 months)118
Downloads (Last 6 weeks)12

Reflects downloads up to 14 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kristensen JBurelli P(2024)Difficulty Modelling in Mobile Puzzle GamesInternational Journal of Computer Games Technology10.1155/2024/55923732024Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1155/2024/5592373
Pfau JSeif El-Nasr M(2024)On Video Game Balancing: Joining Player- and Data-Driven AnalyticsGames: Research and Practice10.1145/36758072:3(1-30)Online publication date: 30-Aug-2024
https://dl.acm.org/doi/10.1145/3675807
Pfau JCharan MKleinman ESeif El-Nasr M(2024)Damage Optimization in Video Games: A Player-Driven Co-Creative ApproachProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642643(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642643
Gonçalves DBarros DPais PGuerreiro JGuerreiro TRodrigues A(2024)The Trick is to Stay Behind?: Defining and Exploring the Design Space of Player Balancing MechanicsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642441(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642441
Pfau J(2024)The Real MVP: Quantifying Individual Performances in Multiplayer Online Games2024 IEEE Conference on Games (CoG)10.1109/CoG60054.2024.10645665(1-8)Online publication date: 5-Aug-2024
https://doi.org/10.1109/CoG60054.2024.10645665
Guo ZThawonmas RRen X(2024)Rethinking dynamic difficulty adjustment for video game designEntertainment Computing10.1016/j.entcom.2024.10066350(100663)Online publication date: May-2024
https://doi.org/10.1016/j.entcom.2024.100663
Munderich LGöbel S(2024)Game On: Towards Long-Term Motivation in Exergames for Cardio TrainingSerious Games10.1007/978-3-031-74138-8_22(315-329)Online publication date: 7-Nov-2024
https://dl.acm.org/doi/10.1007/978-3-031-74138-8_22
Freitas ÉPanceri JSchreider SCaldeira EBastos-Filho T(2024)Memô the Game: Serious Game for Development of Memorization in Children with Autistic Spectrum DisorderIX Latin American Congress on Biomedical Engineering and XXVIII Brazilian Congress on Biomedical Engineering10.1007/978-3-031-49407-9_56(562-572)Online publication date: 4-Jan-2024
https://doi.org/10.1007/978-3-031-49407-9_56
Bicalho LBaffa AFeijó B(2023)A Dynamic Difficulty Adjustment Algorithm With Generic Player Behavior Classification Unity Plugin In Single Player GamesProceedings of the 22nd Brazilian Symposium on Games and Digital Entertainment10.1145/3631085.3631286(76-85)Online publication date: 6-Nov-2023
https://dl.acm.org/doi/10.1145/3631085.3631286
Pfau JSeif El-Nasr M(2023)Balancing Video Games: A Player-Driven InstrumentCompanion Proceedings of the Annual Symposium on Computer-Human Interaction in Play10.1145/3573382.3616097(187-195)Online publication date: 6-Oct-2023
https://dl.acm.org/doi/10.1145/3573382.3616097
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents