research-article

Adaptive Review for Mobile MOOC Learning via Multimodal Physiological Signal Sensing - A Longitudinal Study

Authors:

Jingtao WangAuthors Info & Claims

ICMI '18: Proceedings of the 20th ACM International Conference on Multimodal Interaction

Pages 63 - 72

https://doi.org/10.1145/3242969.3243002

Published: 02 October 2018 Publication History

Abstract

Despite the great potential, Massive Open Online Courses (MOOCs) face major challenges such as low retention rate, limited feedback, and lack of personalization. In this paper, we report the results of a longitudinal study on AttentiveReview2, a multimodal intelligent tutoring system optimized for MOOC learning on unmodified mobile devices. AttentiveReview2 continuously monitors learners' physiological signals, facial expressions, and touch interactions during learning and recommends personalized review materials by predicting each learner's perceived difficulty on each learning topic. In a 3-week study involving 28 learners, we found that AttentiveReview2 on average improved learning gains by 21.8% in weekly tests. Follow-up analysis shows that multi-modal signals collected from the learning process can also benefit instructors by providing rich and fine-grained insights on the learning progress. Taking advantage of such signals also improves prediction accuracies in emotion and test scores when compared with clickstream analysis.

Supplementary Material

suppl.mov (icmi1123.mp4)

Supplemental video

Download
5.13 MB

References

[1]

Afergan, D., Peck, E.M., Solovey, E.T., Jenkins, A., Hincks, S.W., Brown, E.T., Chang, R. and Jacob, R.J., 2014, April. Dynamic difficulty using brain metrics of workload. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems (pp. 3797--3806). ACM.

Digital Library

[2]

Bangor, A., Kortum, P. and Miller, J., 2009. Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of usability studies, 4(3), pp.114--123.

Digital Library

[3]

Brinton, C.G., Rill, R., Ha, S., Chiang, M., Smith, R. and Ju, W., 2015. Individualization for education at scale: MIIC design and preliminary evaluation. IEEE Transactions on Learning Technologies, 8(1), pp.136--148.

Digital Library

[4]

Brooke, J., 1996. SUS-A quick and dirty usability scale. Usability evaluation in industry, 189(194), pp.4--7.

[5]

Chaiklin, S., 2003. The zone of proximal development in Vygotsky's analysis of learning and instruction. Vygotsky's educational theory in cultural context, 1, pp.39--64.

[6]

Chen, C., Alcorn, B., Christensen, G., Eriksson, N., Koller, D. and Emanuel, E., 2015. Who's benefiting from MOOCs, and Why. Harvard Business Review, 22.

[7]

Chuang, I. and Ho, A.D., 2016. HarvardX and MITx: Four Years of Open Online Courses--Fall 2012-Summer 2016.

[8]

Coetzee, D., Fox, A., Hearst, M.A. and Hartmann, B., 2014, March. Chatrooms in MOOCs: all talk and no action. In Proceedings of the first ACM conference on Learning@ scale conference (pp. 127--136). ACM.

Digital Library

[9]

D'Mello, S.K. and Graesser, A., 2010. Multimodal semi-automated affect detection from conversational cues, gross body language, and facial features. User Modeling and User-Adapted Interaction, 20(2), pp.147--187.

Digital Library

[10]

D'Mello, S. and Graesser, A., 2012. Dynamics of affective states during complex learning. Learning and Instruction, 22(2), pp.145--157.

[11]

D'Mello, S., Kopp, K., Bixler, R.E. and Bosch, N., 2016, May. Attending to attention: Detecting and combating mind wandering during computerized reading. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (pp. 1661--1669). ACM.

Digital Library

[12]

Dhawal Shah. 2018. By The Numbers: MOOCS in 2017. Retrieved April 2018 from https://www.class-central.com/report/mooc-stats-2017

[13]

Dunlosky, J., Rawson, K.A., Marsh, E.J., Nathan, M.J. and Willingham, D.T., 2013. Improving students' learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14(1), pp.4--58.

[14]

Grafsgaard, J., Wiggins, J.B., Boyer, K.E., Wiebe, E.N. and Lester, J., 2013, July. Automatically recognizing facial expression: Predicting engagement and frustration. In Educational Data Mining 2013.

[15]

Guo, P.J., Kim, J. and Rubin, R., 2014, March. How video production affects student engagement: An empirical study of MOOC videos. In Proceedings of the first ACM conference on Learning@ scale conference (pp. 41--50). ACM.

Digital Library

[16]

Gütl, C., Rizzardini, R.H., Chang, V. and Morales, M., 2014, September. Attrition in MOOC: Lessons learned from drop-out students. In International Workshop on Learning Technology for Education in Cloud (pp. 37--48). Springer, Cham.

[17]

Han, T., Xiao, X., Shi, L., Canny, J. and Wang, J., 2015, April. Balancing accuracy and fun: Designing camera based mobile games for implicit heart rate monitoring. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (pp. 847--856). ACM.

Digital Library

[18]

Hussain, M.S., Calvo, R.A. and Chen, F., 2013. Automatic cognitive load detection from face, physiology, task performance and fusion during affective interference. Interacting with computers, 26(3), pp.256--268.

[19]

Jiang, Y., Baker, R.S., Paquette, L., San Pedro, M. and Heffernan, N.T., 2015, June. Learning, Moment-by-Moment and Over the Long Term. In International Conference on Artificial Intelligence in Education (pp. 654--657). Springer, Cham.

[20]

Karsenti, T., 2013. What the research says. International Journal of Technologies in Higher Education, 10(2), pp.23--37.

[21]

Kim, J., Guo, P.J., Seaton, D.T., Mitros, P., Gajos, K.Z. and Miller, R.C., 2014, March. Understanding in-video dropouts and interaction peaks in online lecture videos. In Proceedings of the first ACM conference on Learning@ scale conference (pp. 31--40). ACM.

Digital Library

[22]

Kovacs, G., 2015, April. QuizCram: A Question-Driven Video Studying Interface. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (pp. 133--138). ACM.

Digital Library

[23]

Krause, M., Mogalle, M., Pohl, H. and Williams, J.J., 2015, March. A playful game changer: Fostering student retention in online education with social gamification. In Proceedings of the Second (2015) ACM Conference on Learning@ Scale (pp. 95--102). ACM.

Digital Library

[24]

Lehman, B. and Graesser, A., 2015, June. To resolve or not to resolve? that is the big question about confusion. In International Conference on Artificial Intelligence in Education (pp. 216--225). Springer, Cham.

[25]

McDuff, Daniel Jonathan. Crowdsourcing affective responses for predicting media effectiveness. PhD diss., Massachusetts Institute of Technology, 2014.

[26]

McDuff, D., Mahmoud, A., Mavadati, M., Amr, M., Turcot, J. and Kaliouby, R.E., 2016, May. AFFDEX SDK: a cross-platform real-time multi-face expression recognition toolkit. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (pp. 3723--3726). ACM.

Digital Library

[27]

Mehta, Deepak, The Future Of Massively Open Online Courses (MOOCs), #OnCampus, Forbes.com, March 23, 2017.

[28]

Miranda, S., Mangione, G.R., Orciuoli, F., Gaeta, M. and Loia, V., 2013, October. Automatic generation of assessment objects and Remedial Works for MOOCs. In Information Technology Based Higher Education and Training (ITHET), 2013 International Conference on (pp. 1--8). IEEE.

[29]

Monkaresi, H., Bosch, N., Calvo, R.A. and D'Mello, S.K., 2017. Automated detection of engagement using video-based estimation of facial expressions and heart rate. IEEE Transactions on Affective Computing, 8(1), pp.15--28.

Digital Library

[30]

Oviatt, Sharon. The design of future educational interfaces. Routledge, 2013.

[31]

Pham, P. and Wang, J., 2016, October. Adaptive review for mobile MOOC learning via implicit physiological signal sensing. In Proceedings of the 18th ACM International Conference on Multimodal Interaction (pp. 37--44). ACM.

Digital Library

[32]

Pham, P. and Wang, J., 2015, June. AttentiveLearner: improving mobile MOOC learning via implicit heart rate tracking. In International Conference on Artificial Intelligence in Education (pp. 367--376). Springer, Cham.

[33]

Pham, P. and Wang, J., 2017, June. AttentiveLearner2: A Multimodal Approach for Improving MOOC Learning on Mobile Devices. In International Conference on Artificial Intelligence in Education (pp. 561--564). Springer, Cham.

[34]

Picard, Rosalind. Affective computing. Cambridge: MIT press, 1997.

Digital Library

[35]

Raghuveer, Raghuveer, V.R., Tripathy, B.K., Singh, T. and Khanna, S., 2014, December. Reinforcement learning approach towards effective content recommendation in MOOC environments. In MOOC, Innovation and Technology in Education (MITE), 2014 IEEE International Conference on (pp. 285--289). IEEE.

[36]

Sauro, J., 2011. A practical guide to the system usability scale: Background, benchmarks & best practices. Measuring Usability LLC.

[37]

Szafir, D. and Mutlu, B., 2013, April. ARTFul: adaptive review technology for flipped learning. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1001--1010). ACM.

Digital Library

[38]

Van der Sluis, F., Ginn, J. and Van der Zee, T., 2016, April. Explaining Student Behavior at Scale: The influence of video complexity on student dwelling time. In Proceedings of the Third (2016) ACM Conference on Learning@ Scale (pp. 51--60). ACM.

Digital Library

[39]

Verkoeijen, P.P., Rikers, R.M. and Özsoy, B., 2008. Distributed rereading can hurt the spacing effect in text memory. Applied Cognitive Psychology, 22(5), pp.685--695.

[40]

Weiner, B., 1985. An attributional theory of achievement motivation and emotion. Psychological review, 92(4), p.548.

[41]

Xiao, X. and Wang, J., 2016, October. Context and cognitive state triggered interventions for mobile MOOC learning. In Proceedings of the 18th ACM International Conference on Multimodal Interaction (pp. 378--385). ACM.

Digital Library

[42]

Xiao, X. and Wang, J., 2015, November. Towards attentive, bi-directional MOOC learning on mobile devices. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction (pp. 163--170). ACM.

Digital Library

[43]

Xiao, X., Pham, P. and Wang, J., 2017, June. Dynamics of Affective States During MOOC Learning. In International Conference on Artificial Intelligence in Education (pp. 586--589). Springer, Cham.

[44]

Xiao, X., Han, T. and Wang, J., 2013, December. LensGesture: augmenting mobile interactions with back-of-device finger gestures. In Proceedings of the 15th ACM on International conference on multimodal interaction (pp. 287--294). ACM.

Digital Library

[45]

Yang, D., Sinha, T., Adamson, D. and Rosé, C.P., 2013, December. Turn on, tune in, drop out: Anticipating student dropouts in massive open online courses. In Proceedings of the 2013 NIPS Data-driven education workshop (Vol. 11, p. 14).

Cited By

Lewis MHolloman AHernández Cuevas BCrawford C(2024)Exploring Computational Thinking Perspectives in Black Communities with Physiological Computing2024 Black Issues in Computing Education (BICE)10.1109/BICE60192.2024.00013(27-32)Online publication date: 1-Feb-2024
https://doi.org/10.1109/BICE60192.2024.00013
Zhang SChen YJanaka NHaigh CZhao SOoi W(2023)AdaptReview: Towards Effective Video Review Using Text Summaries and Concept MapsHuman-Computer Interaction – INTERACT 202310.1007/978-3-031-42283-6_25(461-481)Online publication date: 25-Aug-2023
https://doi.org/10.1007/978-3-031-42283-6_25
Chen YGao Q(2023)Visualizing timeline‐anchored comments enhanced social presence and information searching in video‐based learningComputer Applications in Engineering Education10.1002/cae.2264131:5(1306-1320)Online publication date: 26-May-2023
https://doi.org/10.1002/cae.22641
Show More Cited By

Index Terms

Adaptive Review for Mobile MOOC Learning via Multimodal Physiological Signal Sensing - A Longitudinal Study
1. Human-centered computing
  1. Ubiquitous and mobile computing
    1. Ubiquitous and mobile devices
      1. Smartphones

Recommendations

Understanding and Detecting Divided Attention in Mobile MOOC Learning
CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

The emergence of mobile apps for Massive Open Online Courses (MOOCs) allows learners to access quality learning materials at low cost and "to control where, what, how and with whom they learn". Unfortunately, when compared with traditional classroom ...
Adaptive review for mobile MOOC learning via implicit physiological signal sensing
ICMI '16: Proceedings of the 18th ACM International Conference on Multimodal Interaction

Massive Open Online Courses (MOOCs) have the potential to enable high quality knowledge dissemination in large scale at low cost. However, today's MOOCs also suffer from low engagement, uni-directional information flow, and lack of personalization. In ...
Context and cognitive state triggered interventions for mobile MOOC learning
ICMI '16: Proceedings of the 18th ACM International Conference on Multimodal Interaction

We present Context and Cognitive State triggered Feed-Forward (C2F2), an intelligent tutoring system and algorithm, to improve both student engagement and learning efficacy in mobile Massive Open Online Courses (MOOCs). C2F2 infers and responds to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICMI '18: Proceedings of the 20th ACM International Conference on Multimodal Interaction

October 2018

687 pages

ISBN:9781450356923

DOI:10.1145/3242969

General Chairs:
Sidney K. D'Mello
University of Illinois, USA
,
Panayiotis (Panos) Georgiou
University of Southern California, USA
,
Stefan Scherer
University of Southern California, USA
,
Program Chairs:
Emily Mower Provost
University of Michigan, USA
,
Mohammad Soleymani
University of Southern California, USA
,
Marcelo Worsley
Northwestern University, USA

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: Specialist Interest Group in Computer-Human Interaction of the ACM

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 October 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICMI '18

Sponsor:

SIGCHI

ICMI '18: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION

October 16 - 20, 2018

CO, Boulder, USA

Acceptance Rates

ICMI '18 Paper Acceptance Rate 63 of 149 submissions, 42%;

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
332
Total Downloads

Downloads (Last 12 months)23
Downloads (Last 6 weeks)2

Reflects downloads up to 16 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lewis MHolloman AHernández Cuevas BCrawford C(2024)Exploring Computational Thinking Perspectives in Black Communities with Physiological Computing2024 Black Issues in Computing Education (BICE)10.1109/BICE60192.2024.00013(27-32)Online publication date: 1-Feb-2024
https://doi.org/10.1109/BICE60192.2024.00013
Zhang SChen YJanaka NHaigh CZhao SOoi W(2023)AdaptReview: Towards Effective Video Review Using Text Summaries and Concept MapsHuman-Computer Interaction – INTERACT 202310.1007/978-3-031-42283-6_25(461-481)Online publication date: 25-Aug-2023
https://doi.org/10.1007/978-3-031-42283-6_25
Chen YGao Q(2023)Visualizing timeline‐anchored comments enhanced social presence and information searching in video‐based learningComputer Applications in Engineering Education10.1002/cae.2264131:5(1306-1320)Online publication date: 26-May-2023
https://doi.org/10.1002/cae.22641
Zhao KMulet JSorita COriola BSerrano MJouffrais C(2022)Remote Graphic-Based Teaching for Pupils with Visual Impairments: Understanding Current Practices and Co-designing an Accessible Tool with Special Education TeachersProceedings of the ACM on Human-Computer Interaction10.1145/35677336:ISS(538-567)Online publication date: 14-Nov-2022
https://dl.acm.org/doi/10.1145/3567733
Das SChakraborty SMitra BBellogin ABoratto LSantos OArdissono LKnijnenburg B(2022)I Cannot See Students Focusing on My Presentation; Are They Following Me? Continuous Monitoring of Student Engagement through “Stungage”Proceedings of the 30th ACM Conference on User Modeling, Adaptation and Personalization10.1145/3503252.3531307(243-253)Online publication date: 4-Jul-2022
https://dl.acm.org/doi/10.1145/3503252.3531307
Lee MLee MLee HLee HChung SChung S(2021)Are You Watching Me?: The Design of a Video-based Learning Management System Using Learners' History DataArchives of Design Research10.15187/adr.2021.11.34.4.22534:4(225-239)Online publication date: 30-Nov-2021
https://doi.org/10.15187/adr.2021.11.34.4.225
Hutt SKrasich KR. Brockmole JK. D'Mello SKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Breaking out of the Lab: Mitigating Mind Wandering with Gaze-Based Attention-Aware Technology in ClassroomsProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445269(1-14)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445269
Hernandez-Cuevas BCrawford C(2021)A Literature Review of Physiological-Based Mobile Educational SystemsIEEE Transactions on Learning Technologies10.1109/TLT.2021.309831514:3(272-291)Online publication date: 1-Jun-2021
https://doi.org/10.1109/TLT.2021.3098315
Kusumastuti DMeyliana Hidayanto APrabowo H(2021)Models of Adaptive Learning System in MOOC: A Systematic Literature Review2021 9th International Conference on Information and Education Technology (ICIET)10.1109/ICIET51873.2021.9419580(242-246)Online publication date: 27-Mar-2021
https://doi.org/10.1109/ICIET51873.2021.9419580
Katada SOkada SHirano YKomatani KTruong KHeylen DCzerwinski MBerthouze NChetouani MNakano M(2020)Is She Truly Enjoying the Conversation?Proceedings of the 2020 International Conference on Multimodal Interaction10.1145/3382507.3418844(315-323)Online publication date: 21-Oct-2020
https://dl.acm.org/doi/10.1145/3382507.3418844
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents