Abstract
Much research in Intelligent Tutoring Systems has explored how to provide on-demand hints, how they should be used, and what effect they have on student learning and performance. Most of this work relies on hints created by experts and assumes that all help provided by the tutor is correct and of high quality. However, hints may not all be of equal value, especially in open-ended problem solving domains, where context is important. This work argues that hint quality, especially when using data-driven hint generation techniques, is inherently uncertain. We investigate the impact of hint quality on students’ help-seeking behavior in an open-ended programming environment with on-demand hints. Our results suggest that the quality of the first few hints on an assignment is positively associated with future hint use on the same assignment. Initial hint quality also correlates with possible help abuse. These results have important implications for hint design and generation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Complete rubric and dataset available at: http://go.ncsu.edu/aied2017-rubric.
- 2.
The Mann-Whitney U test was used were data were not normally distributed.
References
Aleven, V., Koedinger, K.: Limitations of student control: do students know when they need help? In: International Conference on Intelligent Tutoring Systems, pp. 292–303 (2000)
Aleven, V., Mclaren, B., Roll, I., Koedinger, K.: Toward meta-cognitive tutoring: a model of help seeking with a cognitive tutor. Int. J. Artif. Intell. Educ. 16(2), 101–128 (2006)
Aleven, V., Roll, I., Mclaren, B.M., Koedinger, K.R.: Help helps, but only so much: research on help seeking with intelligent tutoring systems. Int. J. Artif. Intell. Educ. 26(1), 1–19 (2016)
Aleven, V., Stahl, E., Schworm, S., Fischer, F., Wallace, R.: Help seeking and help design in interactive learning environments vincent. Rev. Educ. Res. 73(3), 277–320 (2003)
Anderson, J.R.: Rules of the Mind. Lawrence Erlbaum Associates, Hillsdale (1993)
Barnes, T., Stamper, J.: Toward automatic hint generation for logic proof tutoring using historical student data. In: International Conference on Intelligent Tutoring Systems, pp. 373–382 (2008)
Corbett, A., Anderson, J.: Locus of feedback control in computer-based tutoring: impact on learning rate, achievement and attitudes. In: SIGCHI Conference on Human Computer Interaction, pp. 245–252 (2001)
Garcia, D., Harvey, B., Barnes, T.: The beauty and joy of computing. ACM Inroads 6(4), 71–79 (2015)
Kluger, A.N., Denisi, A.: The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol. Bull. 119(2), 254–284 (1996)
Luckin, R., Du Boulay, B.: Ecolab: the development and evaluation of a Vygotskian design framework. Int. J. Artif. Intell. Educ. 10, 198–220 (1999)
Piech, C., Sahami, M., Huang, J., Guibas, L.: Autonomously generating hints by inferring problem solving policies. In: ACM Conference on Learning @ Scale, pp. 1–10 (2015)
Price, T.W., Dong, Y., Barnes, T.: Generating data-driven hints for open-ended programming. In: International Conference on Educational Data Mining (2016)
Price, T.W., Dong, Y., Lipovac, D.: iSnap: towards intelligent tutoring in novice programming environments. In: ACM Technical Symposium on Computer Science Education (2017)
Rivers, K., Koedinger, K.R.: Data-driven hint generation in vast solution spaces: a self-improving python programming tutor. Int. J. Artif. Intell. Educ. 27(1), 37–64 (2017). https://link.springer.com/article/10.1007/s40593-015-0070-z
Roll, I., Aleven, V., McLaren, B.M., Ryu, E., Baker, R.S.J., Koedinger, K.R.: The help tutor: does metacognitive feedback improve students’ help-seeking actions, skills and learning? In: Ikeda, M., Ashley, K.D., Chan, T.-W. (eds.) ITS 2006. LNCS, vol. 4053, pp. 360–369. Springer, Heidelberg (2006). doi:10.1007/11774303_36
Stamper, J., Eagle, M., Barnes, T., Croy, M.: Experimental evaluation of automatic hint generation for a logic tutor. Int. J. Artif. Intell. Educ. 22(1), 3–17 (2013)
Wood, D.: Scaffolding, contingent tutoring and computer-supported learning. Int. J. Artif. Intell. Educ. 12, 280–292 (2001)
Wood, H., Wood, D.: Help seeking, learning and contingent tutoring. Comput. Educ. 33(2–3), 153–169 (1999)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Price, T.W., Zhi, R., Barnes, T. (2017). Hint Generation Under Uncertainty: The Effect of Hint Quality on Help-Seeking Behavior. In: André, E., Baker, R., Hu, X., Rodrigo, M., du Boulay, B. (eds) Artificial Intelligence in Education. AIED 2017. Lecture Notes in Computer Science(), vol 10331. Springer, Cham. https://doi.org/10.1007/978-3-319-61425-0_26
Download citation
DOI: https://doi.org/10.1007/978-3-319-61425-0_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-61424-3
Online ISBN: 978-3-319-61425-0
eBook Packages: Computer ScienceComputer Science (R0)