Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article

Unobtrusive Assessment of Students' Emotional Engagement during Lectures Using Electrodermal Activity Sensors

Published: 18 September 2018 Publication History

Abstract

Modern wearable devices enable the continuous and unobtrusive monitoring of human physiological parameters, including heart rate and electrodermal activity. Through the definition of adequate models these parameters allow to infer the wellbeing, empathy, or engagement of humans in different contexts. In this paper, we show that off-the-shelf wearable devices can be used to unobtrusively monitor the emotional engagement of students during lectures. We propose the use of several novel features to capture students' momentary engagement and use existing methods to characterize the general arousal of students and their physiological synchrony with the teacher. To evaluate our method we collect a data set that -- after data cleaning -- contains data from 24 students, 9 teachers, and 41 lectures. Our results show that non-engaged students can be identified with high reliability. Using a Support Vector Machine, for instance, we achieve a recall of 81% -- which is a 25 percentage points improvement with respect to a Biased Random classifier. Overall, our findings may inform the design of systems that allow students to self-monitor their engagement and act upon the obtained feedback. Teachers could profit of information about non-engaged students too to perform self-reflection and to devise and evaluate methods to (re-)engage students.

References

[1]
Mary Ainley. 2012. Students' Interest and Engagement in Classroom Activities. In Handbook of Research on Student Engagement. Springer.
[2]
Douglas G Altman. 1990. Practical Statistics for Medical Research. CRC Press.
[3]
Kimberly E Arnold, Brandon Karcher, Casey V Wright, and James McKay. 2017. Student Empowerment, Awareness, and Self-Regulation Through a Quantified-Self Student Tool. In Proceedings of the 7th ACM International Learning Analytics 8 Knowledge Conference (LAK 2017).
[4]
Ivon Arroyo, David G. Cooper, Winslow Burleson, Beverly Park Woolf, Kasia Muldner, and Robert Christopherson. 2009. Emotion Sensors Go To School. In Proceedings of the Conference on Artificial Intelligence in Education: Building Learning Systems That Care: From Knowledge Representation to Affective Modelling (AIED 2009).
[5]
Jorn Bakker, Mykola Pechenizkiy, and Natalia Sidorova. 2011. What's Your Current Stress Level? Detection of Stress Patterns from GSR Sensor Data. In Proceedings of the IEEE International Conference on Data Mining Workshops.
[6]
Christian Beckel, Leyna Sadamori, Thorsten Staake, and Silvia Santini. 2014. Revealing Household Characteristics from Smart Meter Data. Energy.
[7]
Mathias Benedek and Christian Kaernbach. 2010. Decomposition of Skin Conductance Data By Means of Nonnegative Deconvolution. Psychophysiology.
[8]
Christopher M Bishop. 2006. Pattern Recognition and Machine Learning. Springer.
[9]
Brandon Booth, Asem Ali, Ian Bennett, and Shrikanth Narayanan. 2017. Toward Active and Unobtrusive Engagement Assessment of Distance Learners (ACII 2017). In Proceedings of 7th International Conference on Affective Computing and Intelligent Interaction.
[10]
Wolfram Boucsein. 2012. Electrodermal Activity. Springer Science 8 Business Media.
[11]
Saša Branković. 2012. Assessment of Brain Monoaminergic Signaling Through Mathematical Modeling of Skin Conductance Response. In Neuroscience-Dealing With Frontiers. InTech.
[12]
John T Cacioppo, Louis G Tassinary, and Gary Berntson. 2007. Handbook of psychophysiology. Cambridge University Press.
[13]
Ryan Cain and Victor R Lee. 2016. Measuring Electrodermal Activity to Capture Engagement in an Afterschool Maker Program. In Proceedings of the 6th Annual Conference on Creativity and Fabrication in Education (FabLearn 2016). ACM.
[14]
Steven Cantrell and Thomas J Kane. 2013. Ensuring Fair and Reliable Measures of Effective Teaching: Culminating Findings from the MET Project's Three-year Study. MET Project Research Paper.
[15]
Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. 2002. SMOTE: Synthetic Minority Over-sampling Technique. Journal of Artificial Intelligence Research.
[16]
Jingjing Chen, Bin Zhu, Olle Balter, Jianliang Xu, Weiwen Zou, Anders Hedman, Rongchao Chen, and Mengdie Sang. 2017. FishBuddy: Promoting Student Engagement in Self-Paced Learning through Wearable Sensing. In In Proceedings of the IEEE International Conference on Smart Computing (SMARTCOMP 2017).
[17]
Sandra L Christenson, Amy L Reschly, and Cathy Wylie. 2012. Handbook of research on student engagement. Springer Science 8 Business Media.
[18]
Corinna Cortes and Vladimir Vapnik. 1995. Support-Vector Networks. Machine learning.
[19]
Elena Di Lascio, Shkurta Gashi, Danilo Krasic, and Silvia Santini. 2017. In-classroom self-tracking for teachers and students: preliminary findings from a pilot study. In In Proceedings of the Adjunction Publication of the ACM International Joint Conference on Pervasive and Ubiquitous Computing (Ubicomp 2017).
[20]
Jennifer A Fredricks, Phyllis C Blumenfeld, and Alison H Paris. 2004. School Engagement: Potential of the Concept, State of the Evidence. Review of educational research.
[21]
Jennifer A Fredricks and Wendy McColskey. 2012. The Measurement of Student Engagement: A Comparative Analysis of Various Methods and Student Self-report instruments. In Handbook of research on student engagement. Springer.
[22]
Katsuya Fujii, Plivelic Marian, Dav Clark, Yoshi Okamoto, and Jun Rekimoto. 2018. Sync Class: Visualization System for In-Class Student Synchronization. In Proceedings of the 9th Augmented Human International Conference (AH 2018).
[23]
Maurizio Garbarino, Matteo Lai, Dan Bender, Rosalind W Picard, and Simone Tognetti. 2014. Empatica E3--A wearable wireless multisensor device for real-time computerized biofeedback and data acquisition. In Proceedings of the 4th EAI International Conference on Wireless Mobile Communication and Healthcare (MobiHealth 2014).
[24]
Nitesh Goyal and Susan R Fussell. 2017. Intelligent Interruption Management using Electro Dermal Activity based Physiological Sensor for Collaborative Sensemaking. In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT 2017).
[25]
Adrián Colomer Granero, Félix Fuentes-Hurtado, Valery Naranjo Ornedo, Jaime Guixeres Provinciale, Jose M Ausín, and Mariano Alcañiz Raya. 2016. A Comparison of Physiological Signal Analysis Techniques and Classifiers for Automatic Emotional Evaluation of Audiovisual Contents. Fontiers in computational neuroscience.
[26]
Alberto Greco, Gaetano Valenza, Antonio Lanata, Enzo Pasquale Scilingo, and Luca Citi. 2016. cvxEDA: A convex optimization approach to electrodermal activity processing. IEEE Transactions on Biomedical Engineering.
[27]
Mariam Hassib, Stefan Schneegass, Philipp Eiglsperger, Niels Henze, Albrecht Schmidt, and Florian Alt. 2017. EngageMeter: A System for Implicit Audience Engagement Sensing Using Electroencephalography. In Proceedings of the Internationalf Conference on Human Factors in Computing Systems (CHI 2017).
[28]
Rui Henriques, Ana Paiva, and Claudia Antunes. 2013. Accessing Emotion Patterns from Affective Interactions Using Electrodermal Activity. In Proceedings of the International Conference on Affective Computing and Intelligent Interaction Humaine Association (ACII 2013).
[29]
Javier Hernandez, Zicheng Liu, Geoff Hulten, Dave DeBarr, Kyle Krum, and Zhengyou Zhang. 2013. Measuring the Engagement Level of TV Viewers. In Proceedings of the IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG 2013).
[30]
Javier Hernandez, Rob R Morris, and Rosalind W Picard. 2011. Call Center Stress Recognition with Person-specific Models. In Proceedings of the International Conference on Affective Computing and Intelligent Interaction (ACII 2011).
[31]
Javier Hernandez, Ivan Riobo, Agata Rozga, Gregory D Abowd, and Rosalind W Picard. 2014. Using Electrodermal Activity to Recognize Ease of Engagement in Children During Social Interactions. In Proceedings of the ACM Joint Conference on Pervasive and Ubiquitous Computing (Ubicomp 2014).
[32]
Marian L. Houser and Caroline Waldbuesser. 2017. Emotional Contagion in the Classroom: The Impact of Teacher Satisfaction and Confirmation on Perceptions of Student Nonverbal Classroom Behavior. College Teaching.
[33]
Nathalie Japkowicz and Mohak Shah. 2011. Evaluating Learning Algorithms: A Classification Perspective. Cambridge University Press.
[34]
Natasha Jaques, Sara Taylor, Asaph Azaria, Asma Ghandeharioun, Akane Sano, and Rosalind Picard. 2015. Predicting Students' Happiness from Physiology, Phone, Mobility, and Behavioral Data. In Proceedings of the International Conference on Affective Computing and Intelligent Interaction (ACII 2015).
[35]
Celine Latulipe, Erin A Carroll, and Danielle Lottridge. 2011. Love, Hate, Arousal and Engagement: Exploring Audience Responses to Performing Arts. In Proceedings of the International Conference on Human Factors in Computing Systems (CHI 2011).
[36]
Victor R Lee, Joel R Drake, and Jeffrey L Thayne. 2016. Appropriating Quantified Self Technologies to Support Elementary Statistical Teaching and Learning. IEEE Transactions on Learning Technologies.
[37]
Jessica Lin, Eamonn Keogh, Li Wei, and Stefano Lonardi. 2007. Experiencing SAX: a Novel Symbolic Representation of Time Series. Data Mining and knowledge discovery.
[38]
David T Lykken and Peter H Venables. 1971. Direct Measurement of Skin Conductance: A Proposal for Standardization. Psychophysiology.
[39]
Carl D Marci, Jacob Ham, Erin Moran, and Scott P Orr. 2007. Physiologic Correlates of Perceived Therapist Empathy and Social-Emotional Process During Psychotherapy. The Journal of nervous and mental disease.
[40]
João Maroco, Ana Lúcia Maroco, Juliana Alvares Duarte Bonini Campos, and Jennifer A Fredricks. 2016. University Student's Engagement: Development of the University Student Engagement Inventory (USEI). Psicologia: Reflexão e Crítica.
[41]
Claudio Martella, Ekin Gedik, Laura Cabrera-Quiros, Gwenn Englebienne, and Hayley Hung. 2015. How Was It?: Exploiting Smartphone Sensing to Measure Implicit Audience Responses to Live Performances. In Proceedings of the 23rd ACM International Conference on Multimedia (ICME 2015).
[42]
Akhil Mathur, Nicholas D Lane, and Fahim Kawsar. 2016. Engagement-Aware Computing: Modelling User Engagement from Mobile Contexts. In Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing (Ubicomp 2016).
[43]
Karen S. McNeal, Jacob M. Spry, Ritayan Mitra, and Jamie L. Tipton. 2014. Measuring Student Engagement, Knowledge, and Perceptions of Climate Change in an Introductory Environmental Geology Course. Journal of Geoscience Education.
[44]
Abhinav Mehrotra, Veljko Pejovic, Jo Vermeulen, Robert Hendley, and Mirco Musolesi. 2016. My Phone and Me: Understanding People's Receptivity to Mobile Notifications. In Proceedings of the International Conference on Human Factors in Computing Systems (CHI 2016).
[45]
Regalena Melrose. 2006. Why Students Underachieve: What Educators and Parents Can Do About It. R8L Education.
[46]
Hamed Monkaresi, Nigel Bosch, Rafael A. Calvo, and Sydney K. D'Mello. 2017. Automated Detection of Engagement Using Video-Based Estimation of Facial Expressions and Heart Rate. IEEE Transactions on Affective Computing.
[47]
Timothy P Mottet and Steven A Beebe. 2000. Emotional Contagion in the Classroom: An Examination of How Teacher and Student Emotions Are Related. ERIC.
[48]
Kasia Muldner, Michael Wixon, Dovan Rai, Winslow Burleson, Beverly Woolf, and Ivon Arroyo. 2015. Exploring the Impact of a Learning Dashboard on Student Affect. In Proceedings of the International Conference on Artificial Intelligence in Education (AIED 2015).
[49]
Andreas C Müller and Sarah Guido. 2016. Introduction to Machine Learning with Python: a Guide for Data Scientists. O'Reilly Media, Inc.
[50]
Richard V Palumbo, Marisa E Marraccini, Lisa L Weyandt, Oliver Wilder-Smith, Heather A McGee, Siwei Liu, and Matthew S Goodwin. 2017. Interpersonal Autonomic Physiology: A Systematic Review of the Literature. Personality and Social Psychology Review.
[51]
Phuong Pham and Jingtao Wang. 2015. AttentiveLearner: Improving Mobile MOOC Learning via Implicit Heart Rate Tracking. In Proceedings of the International Conference on Artificial Intelligence in Education (AIED 2015).
[52]
Rosalind Picard. 1997. Affective computing. Vol. 252. MIT press Cambridge.
[53]
Martin Pielot, Bruno Cardoso, Kleomenis Katevas, Joan Serrà, Aleksandar Matic, and Nuria Oliver. 2017. Beyond Interruptibility: Predicting Opportune Moments to Engage Mobile Phone Users. In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT 2017).
[54]
Ming-Zher Poh, Nicholas C Swenson, and Rosalind W Picard. 2010. A wearable Sensor for Unobtrusive, Long-term Assessment of Electrodermal Activity. IEEE Transactions on Biomedical Engineering.
[55]
John P Pollak, Phil Adams, and Geri Gay. 2011. PAM: a Photographic Affect Meter for Frequent, in Situ Measurement of Affect. In Proceedings of the ACM International Conference on Human Factors in Computing Systems (CHI 2011).
[56]
Kaśka Porayska-Pomsta, Manolis Mavrikis, Sidney D'Mello, Cristina Conati, and Ryan SJd Baker. 2013. Knowledge Elicitation Methods for Affect Modelling in Education. International Journal of Artificial Intelligence in Education.
[57]
Pavel Pudil, Jana Novovičová, and Josef Kittler. 1994. Floating Search Methods in Feature Selection. Pattern recognition letters.
[58]
Samara Ruiz, Sven Charleer, Maite Urretavizcaya, Joris Klerkx, Isabel Fernández-Castro, and Erik Duval. 2016. Supporting Learning by Considering Emotions: Tracking and Visualization a Case Study. In Proceedings of the 6th ACM International Conference on Learning Analytics 8 Knowledge (LAK 2016).
[59]
Akane Sano and Rosalind W Picard. 2013. Stress Recognition Using Wearable Sensors and Mobile Phones. In Proceedings of the International Conference on Affective Computing and Intelligent Interaction (ACII 2013).
[60]
Yutaka Sasaki. 2007. The Truth of the F-measure. Teach Tutor mater.
[61]
Martin EP Seligman. 2012. Flourish: A visionary new understanding of happiness and wellbeing. Simon and Schuster.
[62]
Fernando Silveira, Brian Eriksson, Anmol Sheth, and Adam Sheppard. 2013. Predicting Audience Responses to Movie Content from Electrodermal Activity Signals. In Proceedings of the ACM international Joint Conference on Pervasive and Ubiquitous Computing (Ubicomp 2013).
[63]
Petr Slovák, Paul Tennent, Stuart Reeves, and Geraldine Fitzpatrick. 2014. Exploring Skin Conductance Synchronisation in Everyday Interactions. In Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational (NordiCHI 2014).
[64]
Vasileios Triglianos, Cesare Pautasso, Alessandro Bozzon, and Claudia Hauff. 2016. Inferring Student Attention with ASQ. In Proceedings of the European Conference on Technology Enhanced Learning (ECTEL 2016).
[65]
Chen Wang and Pablo Cesar. 2014. Do We React in the Same Manner?: Comparing GSR Patterns Across Scenarios. In Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational (NordiCHI 2014).
[66]
Chen Wang and Pablo Cesar. 2015. Physiological Measurement on Students' Engagement in a Distributed Learning Environment. In Proceedings of the 2nd International Conference on Physiological Computing Systems (PhyCS 2015).
[67]
Rui Wang, Weichen Wang, Min SH Aung, Dror Ben-Zeev, Rachel Brian, Andrew T Campbell, Tanzeem Choudhury, Marta Hauser, John Kane, Emily A Scherer, et al. 2017. Predicting Symptom Trajectories of Schizophrenia using Mobile Sensing. In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT 2017).
[68]
J. Whitehill, Z. Serpell, Y-C Lin, A. Foster, and J. Movellan. 2014. The Faces of Engagement: Automatic Recognition of Student Engagement from Facial Expressions. In IEEE Transactions on Affective Computing.
[69]
Yuning Zhang, Maysam Haghdan, and Kevin S Xu. 2017. Unsupervised Motion Artifact Detection in Wrist-measured Electrodermal Activity Data. arXiv preprint arXiv:1707.08287 (2017).

Cited By

View all
  • (2024)Towards Reducing Continuous Emotion Annotation Effort During Video Consumption: A Physiological Response Profiling ApproachProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785698:3(1-32)Online publication date: 9-Sep-2024
  • (2024)Understanding Physiological Responses of Students Over Different CoursesProceedings of the 2024 ACM International Symposium on Wearable Computers10.1145/3675095.3676620(104-110)Online publication date: 5-Oct-2024
  • (2024)Inside Out or Not: Privacy Implications of Emotional DisclosureCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3677598(121-125)Online publication date: 5-Oct-2024
  • Show More Cited By

Index Terms

  1. Unobtrusive Assessment of Students' Emotional Engagement during Lectures Using Electrodermal Activity Sensors

      Recommendations

      Comments

      Please enable JavaScript to view thecomments powered by Disqus.

      Information & Contributors

      Information

      Published In

      cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
      Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies  Volume 2, Issue 3
      September 2018
      1536 pages
      EISSN:2474-9567
      DOI:10.1145/3279953
      Issue’s Table of Contents
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 18 September 2018
      Accepted: 01 September 2018
      Revised: 01 April 2018
      Received: 01 February 2018
      Published in IMWUT Volume 2, Issue 3

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Activity
      2. Electrodermal
      3. Engagement
      4. Students
      5. Wearable

      Qualifiers

      • Research-article
      • Research
      • Refereed

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)229
      • Downloads (Last 6 weeks)35
      Reflects downloads up to 18 Nov 2024

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Towards Reducing Continuous Emotion Annotation Effort During Video Consumption: A Physiological Response Profiling ApproachProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785698:3(1-32)Online publication date: 9-Sep-2024
      • (2024)Understanding Physiological Responses of Students Over Different CoursesProceedings of the 2024 ACM International Symposium on Wearable Computers10.1145/3675095.3676620(104-110)Online publication date: 5-Oct-2024
      • (2024)Inside Out or Not: Privacy Implications of Emotional DisclosureCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3677598(121-125)Online publication date: 5-Oct-2024
      • (2024)Detecting Users' Emotional States during Passive Social Media UseProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596068:2(1-30)Online publication date: 15-May-2024
      • (2024)Body-Area Capacitive or Electric Field Sensing for Human Activity Recognition and Human-Computer InteractionProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435558:1(1-49)Online publication date: 6-Mar-2024
      • (2024)Effecti-Net: A Multimodal Framework and Database for Educational Content Effectiveness AnalysisProceedings of the 14th Learning Analytics and Knowledge Conference10.1145/3636555.3636928(667-677)Online publication date: 18-Mar-2024
      • (2024)High-performing Groups during Children's Collaborative Coding Activities: What Can Multimodal Data Tell Us?Proceedings of the 23rd Annual ACM Interaction Design and Children Conference10.1145/3628516.3655805(533-559)Online publication date: 17-Jun-2024
      • (2024)Design Thinking Activities for K-12 Students: Multi-Modal Data Explanations on Coding PerformanceProceedings of the 23rd Annual ACM Interaction Design and Children Conference10.1145/3628516.3655786(290-306)Online publication date: 17-Jun-2024
      • (2024)Characterizing Information Seeking Processes with Multiple Physiological SignalsProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657793(1006-1017)Online publication date: 10-Jul-2024
      • (2024)ConverSense: An Automated Approach to Assess Patient-Provider Interactions using Social SignalsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641998(1-22)Online publication date: 11-May-2024
      • Show More Cited By

      View Options

      Login options

      Full Access

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media