research-article

Open access

Increasing player engagement, retention and performance through the inclusion of educational content in a citizen science game

Authors:

Rogerio de Leon Pereira,

Olivier Tremblay-SavardAuthors Info & Claims

FDG '21: Proceedings of the 16th International Conference on the Foundations of Digital Games

Article No.: 16, Pages 1 - 12

https://doi.org/10.1145/3472538.3472554

Published: 21 October 2021 Publication History

All formats PDF

Abstract

Citizen science game developers often choose to transform or simplify the scientific problem of interest to make the game accessible to a wider audience, which is critical for collecting enough data. A potential drawback of this design decision is the disconnect that it creates between the players and the science. We are interested in studying ways to re-establish that connection and give back to the players in the form of educational content embedded in those games. Here, we briefly describe a preliminary study focused on analyzing different formats (text, cartoon or video), then we mainly focus on studying ways to present educational material (full versus gradual access) and evaluating their effects on players in a citizen science game. The results indicate that although there is no clear consensus on which format is more appropriate, providing gradual access to the content as a reward for completing levels can increase the engagement, the retention and the performance of players.

Supplementary Material

a16-pereira-supplement (a16-pereira-supplement.pdf)

Supplemental file

Download
3.07 MB

References

[1]

Graham Alvare and Richard Gordon. 2015. CT brush and CancerZap!: two video games for computed tomography dose minimization. Theoretical Biology and Medical Modelling 12, 1 (2015), 7.

[2]

Evalyn Bayle. 1942. The nature and causes of regressive movements in reading. The Journal of Experimental Education 11, 1 (1942), 16–36.

[3]

François Bouchy, Maxime Marmier, and Oliver Turner. 2018. Light-Curve Classifications Scrutinized by UNIGE Astronomers. http://mmos.ch/news/2018/01/26/light-curve-classifications-scrutinized-by-unige-astronomers.html

[4]

Anne Bowser, Derek Hansen, Jennifer Preece, Yurong He, Carol Boston, and Jen Hammock. 2014. Gamifying citizen science: a study of two user groups. In Proceedings of the companion publication of the 17th ACM conference on Computer supported cooperative work & social computing. 137–140.

Digital Library

[5]

Carolin Cardamone, Kevin Schawinski, Marc Sarzi, Steven P Bamford, Nicola Bennert, CM Urry, Chris Lintott, William C Keel, John Parejko, Robert C Nichol, 2009. Galaxy Zoo Green Peas: discovery of a class of compact extremely star-forming galaxies. Monthly Notices of the Royal Astronomical Society 399, 3 (2009), 1191–1205.

[6]

Stanford University Carnegie Mellon University. 2010. eterna. https://eternagame.org/ Accessed: 2020-09-08.

[7]

Stanford University Carnegie Mellon University. 2018. eterna. https://www.youtube.com/watch?v=V2qtNyJ3KQg Accessed: 2020-09-08.

[8]

Russell N Carney and Joel R Levin. 2002. Pictorial illustrations still improve students’ learning from text. Educational psychology review 14, 1 (2002), 5–26.

[9]

CCP. 2017. EVE Online: Project Discovery. https://www.eveonline.com/discovery/

[10]

Xin Chen, Jie Zheng, Zheng Fu, Peng Nan, Yang Zhong, Stefano Lonardi, and Tao Jiang. 2005. Assignment of orthologous genes via genome rearrangement. IEEE/ACM Transactions on Computational Biology and Bioinformatics 2, 4(2005), 302–315.

Digital Library

[11]

Christine Chin and Lay-Yen Teou. 2009. Using concept cartoons in formative assessment: Scaffolding students’ argumentation. International Journal of Science Education 31, 10 (2009), 1307–1332.

[12]

Christine Chin and Lay-Yen Teou. 2010. Formative assessment: Using concept cartoon, pupils’ drawings, and group discussions to tackle children’s ideas about biological inheritance. Journal of Biological Education 44, 3 (2010), 108–115.

[13]

Seth Cooper, Firas Khatib, Adrien Treuille, Janos Barbero, Jeehyung Lee, Michael Beenen, Andrew Leaver-Fay, David Baker, Zoran Popović, 2010. Predicting protein structures with a multiplayer online game. Nature 466, 7307 (2010), 756.

[14]

Web Technology Corp.2014. ComiPo!https://www.comipo.com/en/index.html

[15]

Vickie Curtis. 2015. Motivation to participate in an online citizen science game: A study of Foldit. Science Communication 37, 6 (2015), 723–746.

[16]

Peng Dai, Jeffrey M. Rzeszotarski, Praveen Paritosh, and Ed H. Chi. 2015. And Now for Something Completely Different: Improving Crowdsourcing Workflows with Micro-Diversions. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (Vancouver, BC, Canada) (CSCW ’15). Association for Computing Machinery, New York, NY, USA, 628–638. https://doi.org/10.1145/2675133.2675260

Digital Library

[17]

Rogerio de Leon Pereira, Andrea Bunt, and Olivier Tremblay-Savard. 2019. Promoting Learning and Engagement in Human Computing Games: a Study of Educational Material Formats. (2019).

[18]

Sebastian Deterding, Miguel Sicart, Lennart Nacke, Kenton O’Hara, and Dan Dixon. 2011. Gamification. using game-design elements in non-gaming contexts. In CHI’11 extended abstracts on human factors in computing systems. 2425–2428.

[19]

Mira Dontcheva, Robert R Morris, Joel R Brandt, and Elizabeth M Gerber. 2014. Combining crowdsourcing and learning to improve engagement and performance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 3379–3388.

Digital Library

[20]

Fatma Ekici, Erhan Ekici, and Fatih Aydin. 2007. Utility of Concept Cartoons in Diagnosing and Overcoming Misconceptions Related to Photosynthesis.International Journal of Environmental and Science Education 2, 4(2007), 111–124.

[21]

Eyewire. 2019. Eyewire. https://eyewire.org/ Accessed: 2020-09-08.

[22]

Eyewire. 2019. Into the Brain: The Science of Brain Mapping. https://science.eyewire.org/ Accessed: 2020-09-08.

[23]

Paul Ginns. 2005. Meta-analysis of the modality effect. Learning and instruction 15, 4 (2005), 313–331.

[24]

Andreas Holzinger. 2016. Interactive machine learning for health informatics: when do we need the human-in-the-loop?Brain Informatics 3, 2 (2016), 119–131.

[25]

Facebook Inc. 2004. Facebook. http://www.facebook.com Accessed: 2020-06-01.

[26]

Twitter Inc. 2006. Twitter. http://twitter.com Accessed: 2020-06-01.

[27]

Didem İnel and Ali Günay Balım. 2013. Concept cartoons assisted problem based learning method in science and technology teaching and students’ views. Procedia-Social and Behavioral Sciences 93 (2013), 376–380.

[28]

Sebnem Kandil Ingec. 2008. Use of Concept Cartoons as an Assessment Tool in Physics Education.Online Submission 5, 11 (2008), 47–54.

[29]

Charlene Jennett, Laure Kloetzer, Daniel Schneider, Ioanna Iacovides, Anna Cox, Margaret Gold, Brian Fuchs, Alexandra Eveleigh, Kathleen Methieu, Zoya Ajani, 2016. Motivations, learning and creativity in online citizen science. Journal of Science Communication 15, 3 (2016).

[30]

Alexander Kawrykow, Gary Roumanis, Alfred Kam, Daniel Kwak, Clarence Leung, Chu Wu, Eleyine Zarour, Luis Sarmenta, Mathieu Blanchette, Jérôme Waldispühl, 2012. Phylo: a citizen science approach for improving multiple sequence alignment. PloS one 7, 3 (2012), e31362.

[31]

Brenda Keogh and Stuart Naylor. 1999. Concept cartoons, teaching and learning in science: an evaluation. International Journal of Science Education 21, 4 (1999), 431–446.

[32]

Firas Khatib, Seth Cooper, Michael D Tyka, Kefan Xu, Ilya Makedon, Zoran Popović, and David Baker. 2011. Algorithm discovery by protein folding game players. Proceedings of the National Academy of Sciences 108, 47(2011), 18949–18953.

[33]

M Khine. 2013. Critical analysis of science textbooks. Evaluating instructional effectiveness. Netherlands: Springer (2013).

[34]

Jinseop S Kim, Matthew J Greene, Aleksandar Zlateski, Kisuk Lee, Mark Richardson, Srinivas C Turaga, Michael Purcaro, Matthew Balkam, Amy Robinson, Bardia F Behabadi, 2014. Space–time wiring specificity supports direction selectivity in the retina. Nature 509, 7500 (2014), 331.

[35]

Ian Kinchin. 2004. Investigating students’ beliefs about their preferred role as learners. Educational Research 46, 3 (2004), 301–312.

[36]

Mary Koppal and Ann Caldwell. 2004. Meeting the challenge of science literacy: Project 2061 efforts to improve science education. Cell biology education 3, 1 (2004), 28–30.

[37]

Robert B Kozma. 1991. Learning with media. Review of educational research 61, 2 (1991), 179–211.

[38]

Edith Law, Ming Yin, Joslin Goh, Kevin Chen, Michael A. Terry, and Krzysztof Z. Gajos. 2016. Curiosity Killed the Cat, but Makes Crowdwork Better. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 4098–4110. https://doi.org/10.1145/2858036.2858144

Digital Library

[39]

Doris Jung-Lin Lee, Joanne Lo, Moonhyok Kim, and Eric Paulos. 2016. Crowdclass: Designing classification-based citizen science learning modules. In Fourth AAAI Conference on Human Computation and Crowdsourcing.

[40]

Jeehyung Lee, Wipapat Kladwang, Minjae Lee, Daniel Cantu, Martin Azizyan, Hanjoo Kim, Alex Limpaecher, Snehal Gaikwad, Sungroh Yoon, Adrien Treuille, 2014. RNA design rules from a massive open laboratory. Proceedings of the National Academy of Sciences 111, 6 (2014), 2122–2127.

[41]

Claudia Leopold and Richard E Mayer. 2015. An imagination effect in learning from scientific text.Journal of Educational Psychology 107, 1 (2015), 47.

[42]

W Howard Levie and Richard Lentz. 1982. Effects of text illustrations: A review of research. ECTJ 30, 4 (1982), 195–232.

[43]

Joel R Levin. 1987. On empirically validating functions of pictures in prose. The psychology of illustration 1 (1987), 51–86.

[44]

Chris J Lintott, Kevin Schawinski, Anže Slosar, Kate Land, Steven Bamford, Daniel Thomas, M Jordan Raddick, Robert C Nichol, Alex Szalay, Dan Andreescu, 2008. Galaxy Zoo: morphologies derived from visual inspection of galaxies from the Sloan Digital Sky Survey. Monthly Notices of the Royal Astronomical Society 389, 3 (2008), 1179–1189.

[45]

Thomas W Malone, Robert Calfee, Bill Clancey, Allan Collins, Jamey Friend, George Furnas, and Laura Gould. 1981. Toward a theory of intrinsically instruction* Motivating. (1981).

[46]

Anjum Matin, Mardel Maduro, Rogerio de Leon Pereira, and Olivier Tremblay-Savard. 2020. Effect of Timer, Top Score and Leaderboard on Performance and Motivation in a Human Computing Game. In International Conference on the Foundations of Digital Games. 1–10.

Digital Library

[47]

Richard E Mayer and Roxana Moreno. 1998. A split-attention effect in multimedia learning: Evidence for dual processing systems in working memory.Journal of educational psychology 90, 2 (1998), 312.

[48]

Elisa D Mekler, Florian Brühlmann, Alexandre N Tuch, and Klaus Opwis. 2017. Towards understanding the effects of individual gamification elements on intrinsic motivation and performance. Computers in Human Behavior 71 (2017), 525–534.

Digital Library

[49]

Josh Aaron Miller, Uttkarsh Narayan, Matthew Hantsbarger, Seth Cooper, and Magy Seif El-Nasr. 2019. Expertise and engagement: re-designing citizen science games with players’ minds in mind. In Proceedings of the 14th International Conference on the Foundations of Digital Games. 1–11.

Digital Library

[50]

Lennart E Nacke and Christoph Sebastian Deterding. 2017. The maturing of gamification research. Computers in Human Behaviour(2017), 450–454.

[51]

Stuart Naylor and Brenda Keogh. 2013. Concept Cartoons: what have we learnt?Journal of Turkish Science Education 10, 1 (2013).

[52]

Susan Palmiter and Jay Elkerton. 1993. Animated demonstrations for learning procedural computer-based tasks. Human-Computer Interaction 8, 3 (1993), 193–216.

Digital Library

[53]

Susan Palmiter, Jay Elkerton, and Patricia Baggett. 1991. Animated demonstrations vs written instructions for learning procedural tasks: a preliminary investigation. (1991).

[54]

M Jordan Raddick, Georgia Bracey, Pamela L Gay, Chris J Lintott, Phil Murray, Kevin Schawinski, Alexander S Szalay, and Jan Vandenberg. 2009. Galaxy zoo: Exploring the motivations of citizen science volunteers. arXiv preprint arXiv:0909.2925(2009).

[55]

Ralf Rummer, Judith Schweppe, Anne Fürstenberg, Tina Seufert, and Roland Brünken. 2010. Working memory interference during processing texts and pictures: Implications for the explanation of the modality effect. Applied Cognitive Psychology: The Official Journal of the Society for Applied Research in Memory and Cognition 24, 2 (2010), 164–176.

[56]

Michael Sailer, Jan Ulrich Hense, Sarah Katharina Mayr, and Heinz Mandl. 2017. How gamification motivates: An experimental study of the effects of specific game design elements on psychological need satisfaction. Computers in Human Behavior 69 (2017), 371–380.

Digital Library

[57]

Jesse Schell. 2019. The Art of Game Design: A book of lenses. AK Peters/CRC Press.

[58]

Wolfgang Schnotz. 2002. Commentary: Towards an integrated view of learning from text and visual displays. Educational psychology review 14, 1 (2002), 101–120.

[59]

Wolfgang Schnotz. 2011. Colorful bouquets in multimedia research: A closer look at the modality effect. Zeitschrift für Pädagogische Psychologie (2011).

[60]

SciStarter. 2018. SciStarter. http://scistarter.org Accessed: 2020-06-01.

[61]

Matthew Sexton. 2010. Using Concept Cartoons to Access Student Beliefs about Preferred Approaches to Mathematics Learning and Teaching.Mathematics Education Research Group of Australasia (2010).

[62]

Wayne L Shebilske and L Starling Reid. 1979. Reading eye movements, macro-structure and comprehension processes. In Processing of visible language. Springer, 97–110.

[63]

Joel Shurkin. 2015. Science and Culture: Cartoons to better communicate science. Proceedings of the National Academy of Sciences 112, 38(2015), 11741–11742.

[64]

Akash Singh, Chris Drogaris, Elena Nazarova, Mathieu Blanchette, Jérôme Waldispühl, Anjum Ibna Matin, and Olivier Tremblay-Savard. 2017. A human-computation platform for multi-scale genome analysis. Association for the Advancement of Artificial Intelligence (www. aaai. org). Retrieved from https://www. humancomputation. com/2017/papers/100_human-computation-platform. pdf(2017).

[65]

Sparkol. 2018. VideoScribe. https://www.videoscribe.co/en/

[66]

Philip Stephenson and Paul Warwick. 2002. Using concept cartoons to support progression in students’ understanding of light. Physics Education 37, 2 (2002), 135.

[67]

Devin P Sullivan, Casper F Winsnes, Lovisa Åkesson, Martin Hjelmare, Mikaela Wiking, Rutger Schutten, Linzi Campbell, Hjalti Leifsson, Scott Rhodes, Andie Nordgren, 2018. Deep learning is combined with massive-scale citizen science to improve large-scale image classification. Nature biotechnology(2018).

[68]

McGill University. 2010. Phylo. https://phylo.cs.mcgill.ca/ Accessed: 2020-09-08.

[69]

McGill University. 2016. ColonyB. http://csb.cs.mcgill.ca/colonyb/ Accessed: 2020-09-08.

[70]

Luis Von Ahn. 2013. Duolingo: learn a language for free while helping to translate the web. In Proceedings of the 2013 international conference on Intelligent user interfaces. ACM, 1–2.

Digital Library

[71]

Luis Von Ahn and Laura Dabbish. 2004. Labeling images with a computer game. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 319–326.

Digital Library

Cited By

de Leon Pereira RTremblay-Savard O(2024)FORGE: A Framework for Organizing Rewards in Gamified EnvironmentsGames and Culture10.1177/15554120241241555Online publication date: 31-Mar-2024
https://doi.org/10.1177/15554120241241555
Sun YWang JCheng PZheng LChen LYin J(2024)Cross-Domain-Aware Worker Selection with Training for Crowdsourced Annotation2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00026(249-262)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00026
Althani FMadge CPoesio M(2023)The Onboarding Phase in a Game for Text Labelling: Comparing the Effect of Animated vs. Textual Onboarding on Player Experience and Accuracy2023 IEEE Conference on Games (CoG)10.1109/CoG57401.2023.10333246(1-8)Online publication date: 21-Aug-2023
https://doi.org/10.1109/CoG57401.2023.10333246
Show More Cited By

Recommendations

Exploring Video Game Design and Player Retention- a Longitudinal Case Study
Mindtrek '18: Proceedings of the 22nd International Academic Mindtrek Conference

Lasting player engagement is essential for pushing a video game beyond a successful release. While previous research on retention and engagement in games exists, this has mainly focused on general models and factors. Furthermore, we notice a lack of ...
Player performance and in game advertising retention
ACE '11: Proceedings of the 8th International Conference on Advances in Computer Entertainment Technology

In game advertising or IGA is an increasingly common means of promoting brands and products. This study seeks to understand the effectiveness of in game advertising by understanding player retention of brand messages. The researchers created a ...
Engagement Through Praxis in Educational Game Design: Common Threads

Background. Engagement in praxis, the process of acting and reflecting in a cyclical fashion, fosters the development of critical thinking and problem-solving skills among educational game designers. Resulting from this process, designers can apply ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

FDG '21: Proceedings of the 16th International Conference on the Foundations of Digital Games

August 2021

534 pages

ISBN:9781450384223

DOI:10.1145/3472538

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 October 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Genome Canada

Conference

FDG'21

FDG'21: The 16th International Conference on the Foundations of Digital Games 2021

August 3 - 6, 2021

QC, Montreal, Canada

Acceptance Rates

Overall Acceptance Rate 152 of 415 submissions, 37%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
855
Total Downloads

Downloads (Last 12 months)405
Downloads (Last 6 weeks)63

Reflects downloads up to 28 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

de Leon Pereira RTremblay-Savard O(2024)FORGE: A Framework for Organizing Rewards in Gamified EnvironmentsGames and Culture10.1177/15554120241241555Online publication date: 31-Mar-2024
https://doi.org/10.1177/15554120241241555
Sun YWang JCheng PZheng LChen LYin J(2024)Cross-Domain-Aware Worker Selection with Training for Crowdsourced Annotation2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00026(249-262)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00026
Althani FMadge CPoesio M(2023)The Onboarding Phase in a Game for Text Labelling: Comparing the Effect of Animated vs. Textual Onboarding on Player Experience and Accuracy2023 IEEE Conference on Games (CoG)10.1109/CoG57401.2023.10333246(1-8)Online publication date: 21-Aug-2023
https://doi.org/10.1109/CoG57401.2023.10333246
Lee SMott BVandenberg JSpires HLester J(2023)Examining the Relationship of Gameplay and Learning in a Narrative-Centered Digital Game for Science Education2023 IEEE Conference on Games (CoG)10.1109/CoG57401.2023.10333202(1-8)Online publication date: 21-Aug-2023
https://doi.org/10.1109/CoG57401.2023.10333202

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents