research-article

Stereotype Modeling for Problem-Solving Performance Predictions in MOOCs and Traditional Courses

Authors:

Peter Brusilovsky,

Michael Yudelson,

Arto HellasAuthors Info & Claims

UMAP '17: Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization

Pages 76 - 84

https://doi.org/10.1145/3079628.3079672

Published: 09 July 2017 Publication History

Abstract

Stereotypes are frequently used in real life to classify students according to their performance in class. In literature, we can find many references to weaker students, fast learners, struggling students, etc. Given the lack of detailed data about students, these or other kinds of stereotypes could be potentially used for user modeling and personalization in the educational context. Recent research in MOOC context demonstrated that data-driven learner stereotypes could work well for detecting and preventing student dropouts. In this paper, we are exploring the application of stereotype-based modeling to a more challenging task -- predicting student problem-solving and learning in two programming courses and two MOOCs. We explore traditional stereotypes based on readily available factors like gender or education level as well as some advanced data-driven approaches to group students based on their problem-solving behavior. Each of the approaches to form student stereotype cohorts is validated by comparing models of student learning: do students in different groups learn differently? In the search for the stereotypes that could be used for adaptation, the paper examines ten approaches. We compare the performance of these approaches and draw conclusions for future research.

References

[1]

Ashton Anderson, Daniel Huttenlocher, Jon Kleinberg, and Jure Leskovec. 2014. Engaging with massive online courses. In World Wide Web Conf. 687--698.

Digital Library

[2]

Jay Ayres, Jason Flannick, Johannes Gehrke, and Tomi Yiu. 2002. Sequential pattern mining using a bitmap representation. In Knowledge Discovery and Data Mining Conf. 429--435.

Digital Library

[3]

Lori Breslow, David E Pritchard, Jennifer DeBoer, Glenda S Stump, Andrew D Ho, and Daniel T Seaton. 2013. Studying learning in the worldwide classroom: Research into edX's first MOOC. Research & Practice in Assessment 8 (2013), 13--25.

[4]

Peter Brusilovsky and Eva Millan. 2007. User models for adaptive hypermedia and adaptive educational systems. In The Adaptive Web: Methods and Strategies of Web Personalization, Peter Brusilovsky, Alfred Kobsa, and Wolfgang Neidl (Eds.). Lecture Notes in Computer Science, Vol. 4321. Springer-Verlag, Berlin Heidelberg New York, 3--53.

Digital Library

[5]

John Champaign, Kimberly F Colvin, Alwina Liu, Colin Fredericks, Daniel Seaton, and David E Pritchard. 2014. Correlating skill and improvement in 2 MOOCs with a student's time on tasks. In Learning@scale Conf. 11--20.

Digital Library

[6]

Albert T. Corbett and John R. Anderson. 1995. Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction 4, 4 (1995), 253--278.

[7]

Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. 2008. LIBLINEAR: A library for large linear classification. Journal of machine learning research 9 (2008), 1871--1874.

Digital Library

[8]

Philippe Fournier-Viger, Jerry Chun-Wei Lin, Antonio Gomariz, Ted Gueniche, Azadeh Soltani, Zhihong Deng, and Hoang Thanh Lam. 2016. The SPMF OpenSource Data Mining Library Version 2. In Joint European Conf. on Machine Learning and Knowledge Discovery in Databases. 36--40.

[9]

Elena L Glassman, Jeremy Scott, Rishabh Singh, Philip J Guo, and Robert C Miller. 2015. OverCode: Visualizing variation in student solutions to programming problems at scale. Computer-Human Interaction Transaction 22, 2 (2015), 7.

Digital Library

[10]

Julio Guerra, Shaghayegh Sahebi, Yu-Ru Lin, and Peter Brusilovsky. 2014. The problem solving genome: Analyzing sequential patterns of student work with parameterized exercises. In Educational Data Mining Conf. 153--160.

[11]

Roya Hosseini and Peter Brusilovsky. 2013. JavaParser: A Fine-Grained Concept Indexing Tool for Java Problems. In AI-supported Education for Computer Science Workshop. 60--63.

[12]

Roya Hosseini, Arto Vihavainen, and Peter Brusilovsky. 2014. Exploring Problem Solving Paths in a Java Programming Course. In Psychology of Programming Interest Group Conf. 65--76.

[13]

Jonathan Huang, Chris Piech, Andy Nguyen, and Leonidas Guibas. 2013. Syntactic and functional variability of a million code submissions in a machine learning mooc. In AIED 2013 Workshops Proceedings Volume. 25--32.

[14]

Judy Kay. 1994. Lies, damned lies and stereotypes: pragmatic approximations of users. In Fourth International Conference on User Modeling, Alfred Kobsa and Diane Litman (Eds.). MITRE, 175--184.

[15]

René F Kizilcec, Chris Piech, and Emily Schneider. 2013. Deconstructing disen-gagement: analyzing learner subpopulations in massive open online courses. In Proceedings of the third international conference on learning analytics and knowledge. ACM, 170--179.

Digital Library

[16]

Christian Murphy, Gail Kaiser, Kristin Loveland, and Sahar Hasan. 2009. Retina: helping students and instructors based on observed programming activities. ACM SIGCSE Bulletin 41, 1 (2009), 178--182.

Digital Library

[17]

Fred GWC Paas and Jeroen JG Van Merriënboer. 1993. The efficiency of in-structional conditions: An approach to combine mental effort and performance measures. Human Factors: The Journal of the Human Factors and Ergonomics Society 35, 4 (1993), 737--743.

[18]

Philip I Pavlik Jr, Hao Cen, and Kenneth R Koedinger. 2009. Performance Factors Analysis -- A New Alternative to Knowledge Tracing. In Artificial Intelligence in Education Conf. 121--130.

Digital Library

[19]

David N. Perkins, Chris Hancock, Renee Hobbs, Fay Martin, and Rebecca Simmons. 1986. Conditions of learning in novice programmers. Journal of Educational Computing Research 2, 1 (1986), 37--55.

[20]

Chris Piech, Mehran Sahami, Daphne Koller, Steve Cooper, and Paulo Blikstein. 2012. Modeling how students learn to program. In ACM Technical Symposium on Computer Science Education. 153--160.

Digital Library

[21]

Elaine A. Rich. 1983. Users are individuals: individualizing user models. International Journal on the Man-Machine Studies 18 (1983), 199--214.

[22]

Kelly Rivers and Kenneth R Koedinger. 2013. Automatic generation of programming feedback: A data-driven approach. In AI-supported Education for Computer Science Workshop, Vol. 50--59.

[23]

Kshitij Sharma, Patrick Jermann, and Pierre Dillenbourg. 2015. Identifying Styles and Paths toward Success in MOOCs. In Educational Data Mining Conf. 408--411.

[24]

Krisztina Tóth, Heiko Rölke, Samuel Greiff, and Sascha Wüstenberg. 2014. Discov-ering Students' Complex Problem Solving Strategies in Educational Assessment. In Educational Data Mining Conf. 225--228.

[25]

Arto Vihavainen, Thomas Vikberg, Matti Luukkainen, and Martin Pärtel. 2013. Scaffolding Students' Learning Using Test My Code. In Innovation and Technology in Computer Science Education. 117--122.

Digital Library

[26]

Xu Wang, Diyi Yang, Miaomiao Wen, Kenneth Koedinger, and Carolyn P Rosé. 2015. Investigating How Student's Cognitive Behavior in MOOC Discussion Forums Affect Learning Gains. In Educational Data Mining Conf. 226--233.

[27]

Miaomiao Wen and Carolyn Penstein Rosé. 2014. Identifying latent study habits by mining learner behavior patterns in massive open online courses. In Conf. on Information and Knowledge Management. 1983--1986.

Digital Library

[28]

Michael Yudelson, Steve Fancsali, Steve Ritter, Susan Berman, Tristan Nixon, and Ambarish Joshi. 2014. BeTher Data Beats Big Data. In Educational Data Mining Conf. 205--208.

[29]

Michael Yudelson, Roya Hosseini, Arto Vihavainen, and Peter Brusilovsky. 2014. Investigating automated student modeling in a Java MOOC. In Educational Data Mining Conf. 261--264.

Cited By

Kong BHemberg EBell AO'Reilly UHilliger IKhosravi HRienties BDawson S(2023)Investigating Student's Problem-solving Approaches in MOOCs using Natural Language ProcessingLAK23: 13th International Learning Analytics and Knowledge Conference10.1145/3576050.3576091(262-272)Online publication date: 13-Mar-2023
https://dl.acm.org/doi/10.1145/3576050.3576091
Strousopoulos PTroussas CKrouska AMylonas PSgouropoulou C(2023)Integrating Stereotype User Models for Adaptive Scenarios in Game Playing within Immersive Virtual Environments2023 18th International Workshop on Semantic and Social Media Adaptation & Personalization (SMAP)18th International Workshop on Semantic and Social Media Adaptation & Personalization (SMAP 2023)10.1109/SMAP59435.2023.10255160(1-6)Online publication date: 25-Sep-2023
https://doi.org/10.1109/SMAP59435.2023.10255160
Malmi LHellas AIhantola PIsomöttönen VJormanainen IKilamo TKnutas AKorhonen ALaakso MLópez-Pernas SPoranen TSalakoski TSuhonen J(2023)Computing Education Research in FinlandPast, Present and Future of Computing Education Research10.1007/978-3-031-25336-2_16(335-372)Online publication date: 5-Jan-2023
https://doi.org/10.1007/978-3-031-25336-2_16
Show More Cited By

Index Terms

Stereotype Modeling for Problem-Solving Performance Predictions in MOOCs and Traditional Courses

Recommendations

Supporting learners' self-regulated learning in Massive Open Online Courses
Abstract
In MOOCs, learners are typically presented with great autonomy over their learning process. Therefore, learners should engage in self-regulated learning (SRL) in order to successfully study in a MOOC. Learners however often struggle to self-...
Highlights
- Learners struggle to regulate their learning in massive open online courses (MOOCs).
- A self-regulated learning (SRL) intervention was implemented in three MOOCs.
- Learners' SRL was measured with trace data variables.
- ...
Crowd-sourced learning in MOOCs: learning analytics meets measurement theory
LAK '15: Proceedings of the Fifth International Conference on Learning Analytics And Knowledge

This paper illustrated the promise of the combination of measurement theory and learning analytics for understanding effective MOOC learning. It reports findings from a study of whether and how MOOC log file data can assist in understanding how MOOC ...
Facilitating MOOCs learning through weekly meet-up: a case study in Taiwan
L@S '14: Proceedings of the first ACM conference on Learning @ scale conference

Online learners need various supports to survive, and it is especially true in the context of MOOCs. Yet, studies documenting the learning progress as a function of learner support are at its inception. Based on self-determination theory and via weekly ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

UMAP '17: Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization

July 2017

420 pages

ISBN:9781450346351

DOI:10.1145/3079628

General Chairs:
Maria Bielikova
Slovak University of Technology in Bratislava, Slovakia
,
Eelco Herder
L3S Research Center, Germany and Radboud Universiteit Nijmegen, The Netherlands
,
Program Chairs:
Federica Cena
University of Torino, Italy
,
Michel Desmarais
Polytechnique University, Montreal, Canada

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 July 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

UMAP '17

Sponsor:

UMAP '17: 25th Conference on User Modeling, Adaptation and Personalization

July 9 - 12, 2017

Bratislava, Slovakia

Acceptance Rates

UMAP '17 Paper Acceptance Rate 29 of 80 submissions, 36%;

Overall Acceptance Rate 162 of 633 submissions, 26%

Upcoming Conference

UMAP '25

Sponsor:
sigchi
sigchi

33rd ACM Conference on User Modeling, Adaptation and Personalization

June 16 - 19, 2025

New York City , NY , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
423
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)4

Reflects downloads up to 17 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kong BHemberg EBell AO'Reilly UHilliger IKhosravi HRienties BDawson S(2023)Investigating Student's Problem-solving Approaches in MOOCs using Natural Language ProcessingLAK23: 13th International Learning Analytics and Knowledge Conference10.1145/3576050.3576091(262-272)Online publication date: 13-Mar-2023
https://dl.acm.org/doi/10.1145/3576050.3576091
Strousopoulos PTroussas CKrouska AMylonas PSgouropoulou C(2023)Integrating Stereotype User Models for Adaptive Scenarios in Game Playing within Immersive Virtual Environments2023 18th International Workshop on Semantic and Social Media Adaptation & Personalization (SMAP)18th International Workshop on Semantic and Social Media Adaptation & Personalization (SMAP 2023)10.1109/SMAP59435.2023.10255160(1-6)Online publication date: 25-Sep-2023
https://doi.org/10.1109/SMAP59435.2023.10255160
Malmi LHellas AIhantola PIsomöttönen VJormanainen IKilamo TKnutas AKorhonen ALaakso MLópez-Pernas SPoranen TSalakoski TSuhonen J(2023)Computing Education Research in FinlandPast, Present and Future of Computing Education Research10.1007/978-3-031-25336-2_16(335-372)Online publication date: 5-Jan-2023
https://doi.org/10.1007/978-3-031-25336-2_16
Akhuseyinoglu KBrusilovsky P(2022)Exploring Behavioral Patterns for Data-Driven Modeling of Learners' Individual DifferencesFrontiers in Artificial Intelligence10.3389/frai.2022.8073205Online publication date: 15-Feb-2022
https://doi.org/10.3389/frai.2022.807320
Trakunphutthirak RLee V(2021)Application of Educational Data Mining Approach for Student Academic Performance Prediction Using Progressive Temporal DataJournal of Educational Computing Research10.1177/0735633121104877760:3(742-776)Online publication date: 27-Sep-2021
https://doi.org/10.1177/07356331211048777
Lorås MSindre GTrætteberg HAalberg T(2021)Study Behavior in Computing Education—A Systematic Literature ReviewACM Transactions on Computing Education10.1145/346912922:1(1-40)Online publication date: 18-Oct-2021
https://dl.acm.org/doi/10.1145/3469129
Akhuseyinoglu KBrusilovsky P(2021)Data-Driven Modeling of Learners’ Individual Differences for Predicting Engagement and Success in Online LearningProceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization10.1145/3450613.3456834(201-212)Online publication date: 21-Jun-2021
https://dl.acm.org/doi/10.1145/3450613.3456834
Morshed Fahid FTian XEmerson AB. Wiggins JBounajim DSmith AWiebe EMott BElizabeth Boyer KLester J(2021)Progression Trajectory-Based Student Modeling for Novice Block-Based ProgrammingProceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization10.1145/3450613.3456833(189-200)Online publication date: 21-Jun-2021
https://dl.acm.org/doi/10.1145/3450613.3456833
Lei PMendes A(2021)A systematic literature review on knowledge tracing in learning programming2021 IEEE Frontiers in Education Conference (FIE)10.1109/FIE49875.2021.9637323(1-7)Online publication date: 13-Oct-2021
https://doi.org/10.1109/FIE49875.2021.9637323
Troussas CChrysafiadi KVirvou M(2021)Personalized tutoring through a stereotype student model incorporating a hybrid learning style instrumentEducation and Information Technologies10.1007/s10639-020-10366-226:2(2295-2307)Online publication date: 1-Mar-2021
https://dl.acm.org/doi/10.1007/s10639-020-10366-2
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents