research-article

Open access

Understanding Fun in Learning to Code: A Multi-Modal Data approach

Authors:

Gabriella Tisza,

Kshitij Sharma,

Sofia Papavlasopoulou,

Panos Markopoulos,

Michail GiannakosAuthors Info & Claims

IDC '22: Proceedings of the 21st Annual ACM Interaction Design and Children Conference

Pages 274 - 287

https://doi.org/10.1145/3501712.3529716

Published: 27 June 2022 Publication History

All formats PDF

Abstract

The role of fun in learning, and specifically in learning to code, is critical but not yet fully understood. Fun is typically measured by post session questionnaires, which are coarse-grained, evaluating activities that sometimes last an hour, a day or longer. Here we examine how fun impacts learning during a coding activity, combining continuous physiological response data from wristbands and facial expressions from facial camera videos, along with self-reported measures (i.e. knowledge test and reported fun). Data were collected from primary school students (N = 53) in a single-occasion, two-hours long coding workshop, with the BBC micro:bits. We found that a) sadness, anger and stress are negatively, and arousal is positively related to students’ relative learning gain (RLG), b) experienced fun is positively related to students' RLG and c) RLG and fun are related to certain physiological markers derived from the physiological response data.

References

[1]

Brandon Amos, Bartosz Ludwiczuk, and Mahadev Satyanarayanan. 2016. OpenFace: A general-purpose face recognition library with mobile applications. Technical Report. CMU-CS-16-118, CMU School of Computer Science.

[2]

Alejandro Andrade, Joshua A. Danish, and Adam V. Maltese. 2017. A Measurement Model of Gestures in an Embodied Learning Environment: Accounting for Temporal Dependencies. Journal of Learning Analytics 4, 3. https://doi.org/10.18608/jla.2017.43.3

[3]

Alejandro Andrade, Ginette Delandshere, and Joshua A. Danish. 2016. Using Multimodal Learning Analytics to Model Student Behavior: A Systematic Analysis of Epistemological Framing. Journal of Learning Analytics 3, 2: 282–306. https://doi.org/10.18608/jla.2016.32.14

[4]

L.F. Barrett, R. Adolphs, S. Marsella, A. M. Martinez, and S. D. Pollak. 2019. Emotional Expressions Reconsidered: Challenges to Inferring Emotion From Human Facial Movements. Psychological Science in the Public Interest 20, 3: 165–166. https://doi.org/10.1177/1529100619889954

[5]

Paulo Blikstein and Marcelo Worsley. 2016. Multimodal Learning Analytics and Education Data Mining: using computational technologies to measure complex learning tasks. Journal of Learning Analytics 3, 2: 220–238. https://doi.org/10.18608/jla.2016.32.11

[6]

Benjamin Bloom. 1956. Taxonomy of educational objectives: The classification of educational goals. David McKay Co Inc, New York, NY, US.

[7]

Boyan Bontchev and Dessislava Vassileva. 2016. Assessing engagement in an emotionally-adaptive applied game. In Proceedings of the Fourth International Conference on Technological Ecosystems for Enhancing Multiculturality, 747–754. https://doi.org/10.1145/3012430.3012602

Digital Library

[8]

Chris A. Boulton, Carmel Kent, and Hywel T.P. Williams. 2018. Virtual learning environment engagement and learning outcomes at a ‘bricks-and-mortar’ university. Computers & Education 126: 129–142. https://doi.org/10.1016/j.compedu.2018.06.031

[9]

John T. Cacioppo and Louis G. Tassinary. 1990. Inferring psychological significance from physiological signals. American Psychologist 45, 1: 16–28. https://doi.org/10.1037/0003-066X.45.1.16

[10]

Michelene T. H. Chi and Ruth Wylie. 2014. The ICAP Framework: Linking Cognitive Engagement to Active Learning Outcomes. Educational Psychologist 49, 4: 219–243. https://doi.org/10.1080/00461520.2014.965823

[11]

J. F. Cohn, Z. Ambadar, and P. Ekman. 2007. Observer-based measurement of facial expression with the Facial Action Coding System. e. In The handbook of emotion elicitation and assessment,. 203–221.

[12]

Mutlu Cukurova, Michail Giannakos, and Roberto Martinez‐Maldonado. 2020. The promise and challenges of multimodal learning analytics. British Journal of Educational Technology 51, 5: 1441–1449. https://doi.org/10.1111/bjet.13015

[13]

Mutlu Cukurova, Carmel Kent, and Rosemary Luckin. 2019. Artificial intelligence and multimodal data in the service of human decision‐making: A case study in debate tutoring. British Journal of Educational Technology 50, 6: 3032–3046. https://doi.org/10.1111/bjet.12829

[14]

Sidney D'Mello and Art Graesser. 2012. Dynamics of affective states during complex learning. Learning and Instruction 22, 2: 145–157. https://doi.org/10.1016/j.learninstruc.2011.10.001

[15]

P. Ekman, W. V. Friesen, and J. C. Hager. 2002. Facial Action Coding System. The Manual On CD ROM.

[16]

Paul Ekman. 1992. An argument for basic emotions. Cognition and Emotion 6, 3–4: 169–200. https://doi.org/10.1080/02699939208411068

[17]

Paul Ekman, Robert W Levenson, and Wallace V. Friesen. 1983. Autonomic Nervous System Activity Distinguishes Among Emotions. Science 221, 4616: 1208–1210. https://doi.org/10.1126/science.6612338

[18]

Arthur C. Graesser. 2019. Emotions are the experiential glue of learning environments in the 21st century. Learning and Instruction, May: 101212. https://doi.org/10.1016/j.learninstruc.2019.05.009

[19]

Arthur C. Graesser, Shulan Lu, Brent A. Olde, Elisa Cooper-Pye, and Shannon Whitten. 2005. Question asking and eye tracking during cognitive disequilibrium: Comprehending illustrated texts on devices when the devices break down. Memory & Cognition 33, 7: 1235–1247. https://doi.org/10.3758/BF03193225

[20]

Jianpeng Guo. 2018. Building bridges to student learning: Perceptions of the learning environment, engagement, and learning outcomes among Chinese undergraduates. Studies in Educational Evaluation 59: 195–208. https://doi.org/10.1016/j.stueduc.2018.08.002

[21]

Douglas Heaven. 2020. Why faces don't always tell the truth about feelings. Nature 578, 7796: 502–504. https://doi.org/10.1038/d41586-020-00507-5

[22]

Nina Iten and Dominik Petko. 2016. Learning with serious games: Is fun playing the game a predictor of learning success? British Journal of Educational Technology 47, 1: 151–163. https://doi.org/10.1111/bjet.12226

[23]

Marian Joëls, Zhenwei Pu, Olof Wiegert, Melly S. Oitzl, and Harm J. Krugers. 2006. Learning under stress: how does it work? Trends in Cognitive Sciences 10, 4: 152–158. https://doi.org/10.1016/j.tics.2006.02.002

[24]

Heather Killen, David Weintrop, and Megean Garvin. 2019. AP Computer Science Principles’ Impact on the Landscape of High School Computer Science using Maryland as a Model. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education, 1060–1066. https://doi.org/10.1145/3287324.3287356

Digital Library

[25]

Panagiotis Kosmas, Andri Ioannou, and Symeon Retalis. 2018. Moving Bodies to Moving Minds: A Study of the Use of Motion-Based Games in Special Education. TechTrends 62, 6: 594–601. https://doi.org/10.1007/s11528-018-0294-5

[26]

Elena Di Lascio, Shkurta Gashi, and Silvia Santini. 2018. Unobtrusive Assessment of Students’ Emotional Engagement during Lectures Using Electrodermal Activity Sensors. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 3: 1–21. https://doi.org/10.1145/3264913

Digital Library

[27]

Serena Lee-Cultura, Kshitij Sharma, and Michail Giannakos. 2021. Children's play and problem-solving in motion-based learning technologies using a multi-modal mixed methods approach. International Journal of Child-Computer Interaction: 100355. https://doi.org/10.1016/j.ijcci.2021.100355

Digital Library

[28]

Serena Lee-Cultura, Kshitij Sharma, Sofia Papavlasopoulou, Symeon Retalis, and Michail Giannakos. 2020. Using sensing technologies to explain children's self-representation in motion-based educational games. Proceedings of the Interaction Design and Children Conference, IDC 2020, June: 541–555. https://doi.org/10.1145/3392063.3394419

Digital Library

[29]

Kristina Loderer, Reinhard Pekrun, and James C. Lester. 2020. Beyond cold technology: A systematic review and meta-analysis on emotions in technology-based learning environments. Learning and Instruction 70: 101162. https://doi.org/10.1016/j.learninstruc.2018.08.002

[30]

Ju Long. 2007. Just For Fun: Using Programming Games in Software Programming Training and Education - A Field Study of IBM Robocode Community. Journal of Information Technology Education 6: 279–290. https://doi.org/10.28945/216

[31]

Dorothy Lucardie. 2014. The Impact of Fun and Enjoyment on Adult's Learning. Procedia - Social and Behavioral Sciences 142: 439–446. https://doi.org/10.1016/j.sbspro.2014.07.696

[32]

Shayan Mirjafari, Kizito Masaba, Ted Grover, Weichen Wang, Pino Audia, Andrew T. Campbell, Nitesh V. Chawla, Vedant Das Swain, Munmun De Choudhury, Anind K. Dey, Sidney K. D'Mello, Ge Gao, Julie M. Gregg, Krithika Jagannath, Kaifeng Jiang, Suwen Lin, Qiang Liu, Gloria Mark, Gonzalo J. Martinez, Stephen M. Mattingly, Edward Moskal, Raghu Mulukutla, Subigya Nepal, Kari Nies, Manikanta D. Reddy, Pablo Robles-Granda, Koustuv Saha, Anusha Sirigiri, and Aaron Striegel. 2019. Differentiating Higher and Lower Job Performers in the Workplace Using Mobile Sensing. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 2: 1–24. https://doi.org/10.1145/3328908

Digital Library

[33]

Sofia Papavlasopoulou, Michail N. Giannakos, and Letizia Jaccheri. 2019. Exploring children's learning experience in constructionism-based coding activities through design-based research. Computers in Human Behavior. https://doi.org/10.1016/j.chb.2019.01.008

Digital Library

[34]

Sofia Papavlasopoulou, Kshitij Sharma, Michail Giannakos, and Letizia Jaccheri. 2017. Using Eye-Tracking to Unveil Differences Between Kids and Teens in Coding Activities. In Proceedings of the 2017 Conference on Interaction Design and Children, 171–181. https://doi.org/10.1145/3078072.3079740

Digital Library

[35]

Sofia Papavlasopoulou, Kshitij Sharma, and Michail N. Giannakos. 2018. How do you feel about learning to code? Investigating the effect of children's attitudes towards coding using eye-tracking. International Journal of Child-Computer Interaction 17: 50–60. https://doi.org/10.1016/j.ijcci.2018.01.004

[36]

Sofia Papavlasopoulou, Kshitij Sharma, and Michail N. Giannakos. 2019. Coding activities for children: Coupling eye-tracking with qualitative data to investigate gender differences. Computers in Human Behavior, 7491: 1–11. https://doi.org/10.1016/j.chb.2019.03.003

Digital Library

[37]

Reinhard Pekrun. 2006. The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and practice. Educational Psychology Review 18, 4: 315–341. https://doi.org/10.1007/s10648-006-9029-9

[38]

Janet C. Read. 2008. Validating the Fun Toolkit: an instrument for measuring children's opinions of technology. Cognition, Technology & Work 10, 2: 119–128. https://doi.org/10.1007/s10111-007-0069-9

Digital Library

[39]

Mitchel Resnick. 1998. Technologies for lifelong kindergarten. Educational Technology Research and Development 46, 4: 43–55. https://doi.org/10.1007/BF02299672

[40]

José Manuel Sáez-López, Marcos Román-González, and Esteban Vázquez-Cano. 2016. Visual programming languages integrated across the curriculum in elementary school: A two year case study using “scratch” in five schools. Computers and Education 97: 129–141. https://doi.org/10.1016/j.compedu.2016.03.003

Digital Library

[41]

M. A. Sayette, J. F. Cohn, J. M. Wertz, M. A. Perrott, and D. J. Parrott. 2001. A psychometric evaluation of the facial action coding system for assessing spontaneous expression. Journal of Nonverbal Behavior 25, 3: 167–185.

[42]

K. Sharma, S. Lee-Cultura, and M. Giannakos. 2022. Keep Calm and Don't Carry-Forward: Towards sensor-data driven AI agent to enhance human learning. Frontiers in Artificial Intelligence 198.

[43]

Kshitij Sharma and Michail Giannakos. 2020. Multimodal data capabilities for learning: What can multimodal data tell us about learning? British Journal of Educational Technology 51, 5: 1450–1484. https://doi.org/10.1111/bjet.12993

[44]

Kshitij Sharma, Evangelos Niforatos, Michail Giannakos, and Vassilis Kostakos. 2020. Assessing Cognitive Performance Using Physiological and Facial Features. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 3. https://doi.org/10.1145/3411811

Digital Library

[45]

Kshitij Sharma, Zacharoula Papamitsiou, and Michail Giannakos. 2019. Building pipelines for educational data using AI and multimodal analytics: A “grey‐box” approach. British Journal of Educational Technology, June: bjet.12854. https://doi.org/10.1111/bjet.12854

[46]

G. Sim, S. MacFarlane, and J. C. Read. 2006. All work and no play: Measuring fun, usability, and learning in software for children. Computers & Education 46, 3: 235–248. https://doi.org/10.1016/j.compedu.2005.11.021

[47]

R.E. Snow, L. Corno, and D.N. Jackson. 1996. Individual differences in affective and conative functions. In Handbook of Educational Psychology.

[48]

Daniel Spikol, Katerina Avramides, and Mutlu Cukurova. 2016. Exploring the interplay between human and machine annotated multimodal analytics in hands-on STEM activities. In Proceedings of the Sixth International Learning Analytics and Knowledge Conference, 522–523. https://doi.org/10.1145/1235

[49]

Daniel Spikol, Emanuele Ruffaldi, Giacomo Dabisias, and Mutlu Cukurova. 2018. Supervised machine learning in multimodal learning analytics for estimating success in project-based learning. Journal of Computer Assisted Learning 34, 4: 366–377. https://doi.org/10.1111/jcal.12263

[50]

Christian Szegedy, Sergey Ioffe, and Vincent Vanhoucke. 2016. Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning. arXiv:1602.07261. Retrieved from http://arxiv.org/abs/1602.07261

[51]

Joachim Taelman, S. Vandeput, A. Spaepen, and S. Van Huffel. 2009. Influence of Mental Stress on Heart Rate and Heart Rate Variability. . 1366–1369. https://doi.org/10.1007/978-3-540-89208-3_324

[52]

Gabriella Tisza and Panos Markopoulos. 2021. Understanding the role of fun in learning to code. International Journal of Child-Computer Interaction 28: 100270. https://doi.org/10.1016/j.ijcci.2021.100270

Digital Library

[53]

Gabriella Tisza and Panos Markopoulos. 2021. FunQ: Measuring the fun experience of a learning activity with adolescents. Current Psychology. https://doi.org/10.1007/s12144-021-01484-2

[54]

Gabriella Tisza, Panos Markopoulos, and Tilde M. Bekker. 2020. Learning to code: Interplay of attitude, emotions and fun. Manuscript submitted for publication.

[55]

Gabriella Tisza, Sijie Zhu, and Panos Markopoulos. 2021. Fun to Enhance Learning, Motivation, Self-efficacy, and Intention to Play in DGBL. In IFIP ICEC 2021. 28–45. https://doi.org/10.1007/978-3-030-89394-1_3

Digital Library

[56]

Tzu-Wei Tsai, Hsiao Yu Lo, and Kai-Shao Chen. 2012. An affective computing approach to develop the game-based adaptive learning material for the elementary students. In Proceedings of the 2012 Joint International Conference on Human-Centered Computer Environments - HCCE ’12, 8. https://doi.org/10.1145/2160749.2160752

Digital Library

[57]

Marcelo Worsley and Paulo Blikstein. 2018. A Multimodal Analysis of Making. International Journal of Artificial Intelligence in Education 28, 3: 385–419. https://doi.org/10.1007/s40593-017-0160-1

[58]

Feiran Zhang, Panos Markopoulos, and Tilde Bekker. 2020. Children's Emotions in Design-Based Learning: a Systematic Review. Journal of Science Education and Technology 29, 4: 459–481. https://doi.org/10.1007/s10956-020-09830-y

Cited By

Valguarnera SLandoni M(2024)Exploring the use of an interactive storytelling toy to engage Ukranian child refugees in learning ItalianProceedings of the 23rd Annual ACM Interaction Design and Children Conference10.1145/3628516.3659420(929-933)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3628516.3659420
Zhang FPossaghi ISharma KPapavlasopoulou S(2024)High-performing Groups during Children's Collaborative Coding Activities: What Can Multimodal Data Tell Us?Proceedings of the 23rd Annual ACM Interaction Design and Children Conference10.1145/3628516.3655805(533-559)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3628516.3655805
Fitton DMacKenzie SRead J(2024)Investigating the Impact of Monetization on Children’s Experience With Mobile GamesProceedings of the 23rd Annual ACM Interaction Design and Children Conference10.1145/3628516.3655794(248-258)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3628516.3655794
Show More Cited By

Recommendations

Understanding the role of fun in learning to code
Abstract
There are growing efforts amongst educators and especially researchers in gamification and maker spaces to ensure that learning environments are fun and enjoyable. Accordingly, efforts to evaluate whether students enjoyed a certain ...
A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies

The popularity of computer games has exploded in recent years, yet methods of evaluating user emotional state during play experiences lag far behind. There are few methods of assessing emotional state, and even fewer methods of quantifying emotion ...
The role of fun in learning
CHI PLAY '21: Extended Abstracts of the 2021 Annual Symposium on Computer-Human Interaction in Play

Fun is often considered as an essential element of learning, especially among researchers in gamification and child-computer interaction. Accordingly, enjoyment of a (non-formal) learning activity or game is frequently an important design and evaluation ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

IDC '22: Proceedings of the 21st Annual ACM Interaction Design and Children Conference

June 2022

718 pages

ISBN:9781450391979

DOI:10.1145/3501712

Copyright © 2022 Owner/Author.

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike International 4.0 License.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 June 2022

Check for updates

Badges

Best Paper

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Horizon 2020 Framework Programme

Conference

IDC '22

Sponsor:

SIGCHI

IDC '22: Interaction Design and Children

June 27 - 30, 2022

Braga, Portugal

Acceptance Rates

Overall Acceptance Rate 172 of 578 submissions, 30%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
735
Total Downloads

Downloads (Last 12 months)320
Downloads (Last 6 weeks)42

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Valguarnera SLandoni M(2024)Exploring the use of an interactive storytelling toy to engage Ukranian child refugees in learning ItalianProceedings of the 23rd Annual ACM Interaction Design and Children Conference10.1145/3628516.3659420(929-933)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3628516.3659420
Zhang FPossaghi ISharma KPapavlasopoulou S(2024)High-performing Groups during Children's Collaborative Coding Activities: What Can Multimodal Data Tell Us?Proceedings of the 23rd Annual ACM Interaction Design and Children Conference10.1145/3628516.3655805(533-559)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3628516.3655805
Fitton DMacKenzie SRead J(2024)Investigating the Impact of Monetization on Children’s Experience With Mobile GamesProceedings of the 23rd Annual ACM Interaction Design and Children Conference10.1145/3628516.3655794(248-258)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3628516.3655794
Possaghi IZhang FSharma KPapavlasopoulou S(2024)Design Thinking Activities for K-12 Students: Multi-Modal Data Explanations on Coding PerformanceProceedings of the 23rd Annual ACM Interaction Design and Children Conference10.1145/3628516.3655786(290-306)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3628516.3655786
Sidekerskienė TDamaševičius R(2024)Pedagogical memes: a creative and effective tool for teaching STEM subjectsInternational Journal of Mathematical Education in Science and Technology10.1080/0020739X.2024.2328818(1-31)Online publication date: 26-Mar-2024
https://doi.org/10.1080/0020739X.2024.2328818

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents