Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3576050.3576105acmotherconferencesArticle/Chapter ViewAbstractPublication PageslakConference Proceedingsconference-collections
research-article

Instruction-Embedded Assessment for Reading Ability in Adaptive Mathematics Software

Published: 13 March 2023 Publication History

Abstract

Adaptive educational software is likely to better support broader and more diverse sets of learners by considering more comprehensive views (or models) of such learners. For example, recent work proposed making inferences about “non-math” factors like reading comprehension while students used adaptive software for mathematics to better support and adapt to learners. We build on this proposed approach to more comprehensive learning modeling by providing an empirical basis for making inferences about students’ reading ability from their performance on activities in adaptive software for mathematics. We lay out an approach to predicting middle school students’ reading ability using their performance on activities within Carnegie Learning’s MATHia, a widely used intelligent tutoring system for mathematics. We focus on how performance in an early, introductory activity as an especially powerful place to consider instruction-embedded assessment of non-math factors like reading comprehension to guide adaptation based on factors like reading ability. We close by discussing opportunities to extend this work by focusing on particular knowledge components or skills tracked by MATHia that may provide important “levers” for driving adaptation based on students’ reading ability while they learn and practice mathematics.

References

[1]
Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. https://www.tensorflow.org/ Software available from tensorflow.org.
[2]
John R. Anderson and Albert T. Corbett. 1995. Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction 4 (1995), 253–278.
[3]
Ryan Shaun Baker, Albert T. Corbett, Kenneth R. Koedinger, and Angela Z. Wagner. 2004. Off-Task Behavior in the Cognitive Tutor Classroom: When Students "Game the System". In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vienna, Austria) (CHI ’04). Association for Computing Machinery, New York, NY, USA, 383–390. https://doi.org/10.1145/985692.985741
[4]
Ryan SJD Baker, Sujith Gowda, Michael Wixon, Jessica Kalka, Angela Wagner, Aatish Salvi, Vincent Aleven, Gail Kusbit, Jaclyn Ocumpaugh, and Lisa Rossi. 2012. Sensor-free automated detection of affect in a Cognitive Tutor for Algebra. In Educational Data Mining 2012. International Educational Data Mining Society.
[5]
Christopher Bishop. 2006. Pattern Recognition and Machine Learning. Springer. https://www.microsoft.com/en-us/research/publication/pattern-recognition-machine-learning/
[6]
Benjamin Samuel Bloom. 1968. Learning for Mastery. Instruction and Curriculum. Regional Education Laboratory for the Carolinas and Virginia, Topical Papers and Reprints 1.
[7]
Tianqi Chen and Carlos Guestrin. 2016. XGBoost: A Scalable Tree Boosting System. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (San Francisco, California, USA) (KDD ’16). ACM, New York, NY, USA, 785–794. https://doi.org/10.1145/2939672.2939785
[8]
François Chollet. 2015. Keras. https://github.com/fchollet/keras.
[9]
Corinna Cortes and Vladimir Naumovich Vapnik. 2004. Support-Vector Networks. Machine Learning 20(2004), 273–297.
[10]
Stephen E. Fancsali, Kenneth Holstein, Michael Sandbothe, Steven Ritter, Bruce M. McLaren, and Vincent Aleven. 2020. Towards Practical Detection of Unproductive Struggle. In Artificial Intelligence in Education, Ig Ibert Bittencourt, Mutlu Cukurova, Kasia Muldner, Rose Luckin, and Eva Millán (Eds.). Springer International Publishing, Cham, 92–97.
[11]
Peter Fuentes. 1998. Reading Comprehension in Mathematics. The Clearing House 72, 2 (1998), 81–88. http://www.jstor.org/stable/30189563
[12]
Neil T. Heffernan and Cristina Heffernan. 2014. The ASSISTments Ecosystem: Building a Platform that Brings Scientists and Teachers Together for Minimally Invasive Research on Human Learning and Teaching. International Journal of Artificial Intelligence in Education 24 (2014), 470–497.
[13]
J. D. Hunter. 2007. Matplotlib: A 2D graphics environment. Computing In Science & Engineering 9, 3 (2007), 90–95. https://doi.org/10.1109/MCSE.2007.55
[14]
Eric Jones, Travis Oliphant, Pearu Peterson, 2001. SciPy: Open source scientific tools for Python. http://www.scipy.org/
[15]
Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. arXiv e-prints, Article arXiv:1412.6980 (Dec. 2014), arXiv:1412.6980 pages. arxiv:1412.6980 [cs.LG]
[16]
Kenneth R. Koedinger, Ryan S. Baker, Kyle Cunningham, Alida Skogsholm, Brett Leber, and John Stamper. 2010. Handbook of Educational Data Mining.CRC Press., Boca Raton, FL, Chapter A Data Repository for the EDM community: The PSLC DataShop.
[17]
Kenneth R. Koedinger, Albert T. Corbett, and Charles Perfetti. 2012. The Knowledge-Learning-Instruction Framework: Bridging the Science-Practice Chasm to Enhance Robust Student Learning. Cognitive Science 36, 5 (2012), 757–798. https://doi.org/10.1111/j.1551-6709.2012.01245.x arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.1551-6709.2012.01245.x
[18]
Kenneth R. Koedinger and Mitchell J. Nathan. 2004. The Real Story Behind Story Problems: Effects of Representations on Quantitative Reasoning. Journal of the Learning Sciences 13, 2 (2004), 129–164. https://doi.org/10.1207/s15327809jls1302_1
[19]
Janina Krawitz, Yu-Ping Chang, Kai-Lin Yang, and Stanislaw Schukajlow. 2022. The role of reading comprehension in mathematical modelling: improving the construction of a real-world model and interest in Germany and Taiwan. Educational Studies in Mathematics 109 (2022), 337–359.
[20]
Zachary A. Pardos, R. Baker, Maria Ofelia San Pedro, Sujith M. Gowda, and Supreeth M. Gowda. 2014. Affective States and State Tests: Investigating How Affect and Engagement during the School Year Predict End-of-Year Learning Outcomes.Journal of learning Analytics 1 (2014), 107–128.
[21]
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research 12 (2011), 2825–2830.
[22]
J. Elizabeth Richey, Nikki G. Lobczowski, Paulo F. Carvalho, and Kenneth Koedinger. 2020. Comprehensive Views of Math Learners: A Case for Modeling and Supporting Non-math Factors in Adaptive Math Software. In Artificial Intelligence in Education, Ig Ibert Bittencourt, Mutlu Cukurova, Kasia Muldner, Rose Luckin, and Eva Millán (Eds.). Springer International Publishing, Cham, 460–471.
[23]
Steven Ritter, John R. Anderson, K. Koedinger, and Albert T. Corbett. 2007. Cognitive Tutor: Applied research in mathematics education. Psychonomic Bulletin & Review 14 (2007), 249–255.
[24]
Steven Ritter, Ambarish Joshi, Stephen Fancsali, and Tristan Nixon. 2013. Predicting Standardized Test Scores from Cognitive Tutor Interactions. In EDM.
[25]
Steven Ritter, April Murphy, Stephen E. Fancsali, Vivek Fitkariwala, Nirmal Patel, and J. Derek Lomas. 2020. UpGrade: An open source tool to support A/B testing in educational software. In Proceedings of the First Workshop on Educational A/B Testing at Scale.EdTech Books.
[26]
Steven Ritter, Michael V. Yudelson, Stephen E. Fancsali, and Susan R. Berman. 2016. How Mastery Learning Works at Scale. Proceedings of the Third (2016) ACM Conference on Learning @ Scale (2016).
[27]
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. J. Mach. Learn. Res. 15, 1 (jan 2014), 1929–1958.
[28]
Piia Maria Vilenius‐Tuohimaa, Kaisa Aunola, and Jari‐Erik Nurmi. 2008. The association between mathematical word problems and reading comprehension. Educational Psychology 28, 4 (2008), 409–426. https://doi.org/10.1080/01443410701708228 arXiv:https://doi.org/10.1080/01443410701708228
[29]
Michael Waskom, Olga Botvinnik, Drew O’Kane, Paul Hobson, Saulius Lukauskas, David C Gemperline, Tom Augspurger, Yaroslav Halchenko, John B. Cole, Jordi Warmenhoven, Julian de Ruiter, Cameron Pye, Stephan Hoyer, Jake Vanderplas, Santi Villalba, Gero Kunter, Eric Quintero, Pete Bachant, Marcel Martin, Kyle Meyer, Alistair Miles, Yoav Ram, Tal Yarkoni, Mike Lee Williams, Constantine Evans, Clark Fitzgerald, Brian, Chris Fonnesbeck, Antony Lee, and Adel Qalieh. 2017. mwaskom/seaborn: v0.8.1. https://doi.org/10.5281/zenodo.883859
[30]
Chuankai Zhang, Yanzun Huang, Jingyu Wang, Dongyang Lu, Weiqi Fang, John C. Stamper, Stephen Fancsali, Kenneth Holstein, and Vincent Aleven. 2019. Early Detection of Wheel Spinning: Comparison across Tutors, Models, Features, and Operationalizations. In EDM.
[31]
Guoguo Zheng, Stephen Edward Fancsali, Steven Ritter, and Susan Berman. 2019. Using Instruction-Embedded Formative Assessment to Predict State Summative Test Scores and Achievement Levels in Mathematics. Journal of Learning Analytics 6, 2 (Aug. 2019), 153–174. https://doi.org/10.18608/jla.2019.62.11

Cited By

View all
  • (2024)Hierarchical Dependencies in Classroom Settings Influence Algorithmic Bias MetricsProceedings of the 14th Learning Analytics and Knowledge Conference10.1145/3636555.3636869(210-218)Online publication date: 18-Mar-2024
  • (2024)Rewriting Content with GPT-4 to Support Emerging Readers in Adaptive Mathematics SoftwareInternational Journal of Artificial Intelligence in Education10.1007/s40593-024-00420-2Online publication date: 19-Jul-2024
  • (2023)Rewriting Math Word Problems to Improve Learning Outcomes for Emerging Readers: A Randomized Field Trial in Carnegie Learning’s MATHiaArtificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky10.1007/978-3-031-36336-8_30(200-205)Online publication date: 30-Jun-2023

Index Terms

  1. Instruction-Embedded Assessment for Reading Ability in Adaptive Mathematics Software

      Recommendations

      Comments

      Please enable JavaScript to view thecomments powered by Disqus.

      Information & Contributors

      Information

      Published In

      cover image ACM Other conferences
      LAK2023: LAK23: 13th International Learning Analytics and Knowledge Conference
      March 2023
      692 pages
      ISBN:9781450398657
      DOI:10.1145/3576050
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 13 March 2023

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. assessments
      2. intelligent tutoring systems
      3. machine learning
      4. predictive modeling

      Qualifiers

      • Research-article
      • Research
      • Refereed limited

      Funding Sources

      • U.S. Department of Education Institute of Education Sciences

      Conference

      LAK 2023

      Acceptance Rates

      Overall Acceptance Rate 236 of 782 submissions, 30%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)65
      • Downloads (Last 6 weeks)6
      Reflects downloads up to 10 Nov 2024

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Hierarchical Dependencies in Classroom Settings Influence Algorithmic Bias MetricsProceedings of the 14th Learning Analytics and Knowledge Conference10.1145/3636555.3636869(210-218)Online publication date: 18-Mar-2024
      • (2024)Rewriting Content with GPT-4 to Support Emerging Readers in Adaptive Mathematics SoftwareInternational Journal of Artificial Intelligence in Education10.1007/s40593-024-00420-2Online publication date: 19-Jul-2024
      • (2023)Rewriting Math Word Problems to Improve Learning Outcomes for Emerging Readers: A Randomized Field Trial in Carnegie Learning’s MATHiaArtificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky10.1007/978-3-031-36336-8_30(200-205)Online publication date: 30-Jun-2023

      View Options

      Get Access

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format.

      HTML Format

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media