research-article

When to stop?: towards universal instructional policies

Authors:

Severin Klingler,

Markus GrossAuthors Info & Claims

LAK '16: Proceedings of the Sixth International Conference on Learning Analytics & Knowledge

Pages 289 - 298

https://doi.org/10.1145/2883851.2883961

Published: 25 April 2016 Publication History

Abstract

The adaptivity of intelligent tutoring systems relies on the accuracy of the student model and the design of the instructional policy. Recently an instructional policy has been presented that is compatible with all common student models. In this work we present the next step towards a universal instructional policy. We introduce a new policy that is applicable to an even wider range of student models including DBNs modeling skill topologies and forgetting. We theoretically and empirically compare our policy to previous policies. Using synthetic and real world data sets we show that our policy can effectively handle wheel-spinning students as well as forgetting across a wide range of student models.

References

[1]

R. S. Baker, A. T. Corbett, S. M. Gowda, A. Z. Wagner, B. A. MacLaren, L. R. Kauffman, A. P. Mitchell, and S. Giguere. Contextual Slip and Prediction of Student Performance after Use of an Intelligent Tutor. In Proc. UMAP, 2010.

Digital Library

[2]

R. S. J. D. Baker, A. B. Goldstein, and N. T. Heffernan. Detecting Learning Moment-by-Moment. IJAIED, 21(1-2):5--25, 2011.

Digital Library

[3]

J. E. Beck and Y. Gong. Wheel-spinning: Students who fail to master a skill. In Proc. AIED, pages 431--440, 2013.

[4]

E. Brunskill. Estimating Prerequisite Structure From Noisy Data. In Proc. EDM, 2011.

[5]

H. Cen, K. R. Koedinger, and B. Junker. Is Over Practice Necessary? -Improving Learning Efficiency with the Cognitive Tutor through Educational Data Mining. In Proc. AIED, 2007.

Digital Library

[6]

H. Cen, K. R. Koedinger, and B. Junker. Comparing Two IRT Models for Conjunctive Skills. In Proc. ITS, 2008.

Digital Library

[7]

K.-M. Chang, J. Beck, J. Mostow, and A. Corbett. A Bayes Net Toolkit for Student Modeling in Intelligent Tutoring Systems. In Proc. ITS, pages 104--113, 2006.

Digital Library

[8]

B. Clement, D. Roy, P. Oudeyer, and M. Lopes. Multi-Armed Bandits for Intelligent Tutoring Systems. JEDM, 7, 2015.

[9]

C. Conati, A. Gertner, and K. VanLehn. Using Bayesian Networks to Manage Uncertainty in Student Modeling. UMUAI, 2002.

Digital Library

[10]

A. T. Corbett and J. R. Anderson. Knowledge Tracing: Modeling the Acquisition of Procedural Knowledge. UMUAI, 1994.

[11]

A. Galyardt and I. Goldin. Move your lamp post: Recent data reflects learner knowledge better than older data. JEDM, 7(2):235--278, 2004.

[12]

J. P. González-Brenes and Y. Huang. Your model is predictive - but is it useful? Theoretical and Empirical Considerations of a New Paradigm for Adaptive Tutoring Evaluation. In Proc. EDM, 2015.

[13]

J. P. González-Brenes and J. Mostow. Dynamic Cognitive Tracing: Towards Unified Discovery of Student and Cognitive Models. In Proc. EDM, 2012.

[14]

J. P. González-Brenes and J. Mostow. Topical Hidden Markov Models for Skill Discovery in Tutorial Data. NIPS - Workshop on Personalizing Education With Machine Learning, 2012.

[15]

T. Käser, G.-M. Baschera, J. Kohn, K. Kucian, V. Richtmann, U. Grond, M. Gross, and M. von Aster. Design and evaluation of the computer-based training program Calcularis for enhancing numerical cognition. Front. Psychol., 2013.

[16]

T. Käser, S. Klingler, A. G. Schwing, and M. Gross. Beyond Knowledge Tracing: Modeling Skill Topologies with Bayesian Networks. In Proc. ITS, pages 188--198, 2014.

Digital Library

[17]

T. Käser, A. G. Schwing, T. Hazan, and M. Gross. Computational Education using Latent Structured Prediction. Proc. AISTATS, 2014.

[18]

K. Koedinger, R. Baker, A. Cunningham, K. and Skogsholm, B. Leber, and J. Stamper. A Data Repository for the EDM community: The PSLC DataShop. In C. Romero, S. Ventura, M. Pechenizkiy, and R. Baker, editors, Handbook of Educational Data Mining. CRC Press, Boca Raton, FL, 2010.

[19]

J. I. Lee and E. Brunskill. The Impact on Individualizing Student Models on Necessary Practice Opportunities. In Proc. EDM, pages 118--125, 2012.

[20]

T. Mandel, Y.-E. Liu, S. Levine, E. Brunskill, and Z. Popovic. Offline Policy Evaluation Across Representations with Applications to Educational Games. In Proc. AAMAS, pages 1077--1084, 2014.

Digital Library

[21]

K. Muldner and C. Conati. Evaluating a decision-theoretic approach to tailored example selection. In Proc. IJCAI, pages 483--488, 2007.

Digital Library

[22]

R. C. Murray, K. Vanlehn, and J. Mostow. Looking ahead to select tutorial actions: A decision-theoretic approach. IJAIED, 14(3-4):235--278, 2004.

Digital Library

[23]

Z. A. Pardos and N. T. Heffernan. Modeling Individualization in a Bayesian Networks Implementation of Knowledge Tracing. In Proc. UMAP, 2010.

Digital Library

[24]

Z. A. Pardos, S. Trivedi, N. T. Heffernan, and G. N. Sárközy. Clustered knowledge tracing. In Proc. ITS, 2012.

Digital Library

[25]

P. I. Pavlik, H. Cen, and K. R. Koedinger. Performance Factors Analysis - A New Alternative to Knowledge Tracing. In Proc. AIED, 2009.

Digital Library

[26]

A. N. Rafferty, E. Brunskill, T. L. Griffiths, and P. Shafto. Faster Teaching by POMDP Planning. In Proc. AIED, pages 280--287, 2011.

Digital Library

[27]

S. Ritter, T. K. Harris, T. Nixon, D. Dickison, R. C. Murray, and B. Towle. Reducing the Knowledge Tracing Space. In Proc. EDM, 2009.

[28]

J. Rollinson and E. Brunskill. From Predictive Models to Instructional Policies. In Proc. EDM, 2015.

[29]

Y. Wang and J. Beck. Class vs. Student in a Bayesian Network Student Model. In Proc. AIED, 2013.

[30]

Y. Wang and N. T. Heffernan. The student skill model. In Proc. ITS, 2012.

Digital Library

[31]

M. V. Yudelson, K. R. Koedinger, and G. J. Gordon. Individualized Bayesian Knowledge Tracing Models. In Proc. AIED, 2013.

Cited By

Pelánek R(2024)Adaptive Learning is Hard: Challenges, Nuances, and Trade-offs in ModelingInternational Journal of Artificial Intelligence in Education10.1007/s40593-024-00400-6Online publication date: 21-Mar-2024
https://doi.org/10.1007/s40593-024-00400-6
Pelánek REffenberger TJarušek P(2024)Personalized recommendations for learning activities in online environments: a modular rule-based approachUser Modeling and User-Adapted Interaction10.1007/s11257-024-09396-z34:4(1399-1430)Online publication date: 6-Apr-2024
https://doi.org/10.1007/s11257-024-09396-z
Pelánek R(2021)Adaptive, Intelligent, and Personalized: Navigating the Terminological Maze Behind Educational TechnologyInternational Journal of Artificial Intelligence in Education10.1007/s40593-021-00251-532:1(151-173)Online publication date: 16-Apr-2021
https://doi.org/10.1007/s40593-021-00251-5
Show More Cited By

Index Terms

When to stop?: towards universal instructional policies
1. Applied computing
  1. Education

Recommendations

Towards Detecting Wheel-Spinning: Future Failure in Mastery Learning
L@S '15: Proceedings of the Second (2015) ACM Conference on Learning @ Scale

Wheel-spinning refers to a phenomenon in which a student has spent a considerable amount of time practicing a skill, yet displays little or no progress towards mastery. Wheel-spinning has been shown to be a common problem affecting a significant number ...
Robust Evaluation Matrix: Towards a More Principled Offline Exploration of Instructional Policies
L@S '17: Proceedings of the Fourth (2017) ACM Conference on Learning @ Scale

The gold standard for identifying more effective pedagogical approaches is to perform an experiment. Unfortunately, frequently a hypothesized alternate way of teaching does not yield an improved effect. Given the expense and logistics of each experiment,...
Individualization of Foreign Language Teaching through Adaptive eLearning

Lifelong learning has become an essential part of each profession. For this reason, personalized and adaptive learning has been drawing attention of professionals in the field of formal as well as informal education in the last few years. The effort has ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

LAK '16: Proceedings of the Sixth International Conference on Learning Analytics & Knowledge

April 2016

567 pages

ISBN:9781450341905

DOI:10.1145/2883851

General Chairs:
Dragan Gašević
University of Edinburgh, United Kingdom
,
Grace Lynch
Society for Learning Analytics Research
,
Program Chairs:
Shane Dawson
University of South Australia
,
Hendrik Drachsler
University of the Netherlands
,
Carolyn Penstein Rosé
Carnegie Mellon University

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

LAK '16

LAK '16: 6th International Conference on Learning Analytics and Knowledge

April 25 - 29, 2016

Edinburgh, United Kingdom

Acceptance Rates

LAK '16 Paper Acceptance Rate 36 of 116 submissions, 31%;

Overall Acceptance Rate 236 of 782 submissions, 30%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
355
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)2

Reflects downloads up to 22 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Pelánek R(2024)Adaptive Learning is Hard: Challenges, Nuances, and Trade-offs in ModelingInternational Journal of Artificial Intelligence in Education10.1007/s40593-024-00400-6Online publication date: 21-Mar-2024
https://doi.org/10.1007/s40593-024-00400-6
Pelánek REffenberger TJarušek P(2024)Personalized recommendations for learning activities in online environments: a modular rule-based approachUser Modeling and User-Adapted Interaction10.1007/s11257-024-09396-z34:4(1399-1430)Online publication date: 6-Apr-2024
https://doi.org/10.1007/s11257-024-09396-z
Pelánek R(2021)Adaptive, Intelligent, and Personalized: Navigating the Terminological Maze Behind Educational TechnologyInternational Journal of Artificial Intelligence in Education10.1007/s40593-021-00251-532:1(151-173)Online publication date: 16-Apr-2021
https://doi.org/10.1007/s40593-021-00251-5
Matayoshi JCosyn EUzun H(2021)Are We There Yet? Evaluating the Effectiveness of a Recurrent Neural Network-Based Stopping Algorithm for an Adaptive AssessmentInternational Journal of Artificial Intelligence in Education10.1007/s40593-021-00240-831:2(304-336)Online publication date: 16-Mar-2021
https://doi.org/10.1007/s40593-021-00240-8
Adjei SBaker RBahel V(2021)Seven-Year Longitudinal Implications of Wheel Spinning and Productive PersistenceArtificial Intelligence in Education10.1007/978-3-030-78292-4_2(16-28)Online publication date: 11-Jun-2021
https://doi.org/10.1007/978-3-030-78292-4_2
Botelho AVaratharaj APatikorn TDoherty DAdjei SBeck J(2019)Developing Early Detectors of Student Attrition and Wheel Spinning Using Deep LearningIEEE Transactions on Learning Technologies10.1109/TLT.2019.291216212:2(158-170)Online publication date: 1-Apr-2019
https://doi.org/10.1109/TLT.2019.2912162
Doroudi SAleven VBrunskill E(2019)Where’s the Reward?International Journal of Artificial Intelligence in Education10.1007/s40593-019-00187-x29:4(568-620)Online publication date: 14-Nov-2019
https://doi.org/10.1007/s40593-019-00187-x
Holstein KHong GTegene MMcLaren BAleven VPardo ABartimote-Aufflick KLynch GShum SFerguson RMerceron AOchoa X(2018)The classroom as a dashboardProceedings of the 8th International Conference on Learning Analytics and Knowledge10.1145/3170358.3170377(79-88)Online publication date: 7-Mar-2018
https://dl.acm.org/doi/10.1145/3170358.3170377
Pelánek R(2018)Conceptual Issues in Mastery Criteria: Differentiating Uncertainty and Degrees of KnowledgeArtificial Intelligence in Education10.1007/978-3-319-93843-1_33(450-461)Online publication date: 20-Jun-2018
https://doi.org/10.1007/978-3-319-93843-1_33
Pelánek RŘihák JBielikova MHerder ECena FDesmarais M(2017)Experimental Analysis of Mastery Learning CriteriaProceedings of the 25th Conference on User Modeling, Adaptation and Personalization10.1145/3079628.3079667(156-163)Online publication date: 9-Jul-2017
https://dl.acm.org/doi/10.1145/3079628.3079667
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents