Abstract
Dialogue Acts (DAs) can be used to explain what expert tutors do and what students know during the tutoring process. Most empirical studies adopt the random sampling method to obtain sentence samples for manual annotation of DAs, which are then used to train DA classifiers. However, these studies have paid little attention to sample informativeness, which can reflect the information quantity of the selected samples and inform the extent to which a classifier can learn patterns. Notably, the informativeness level may vary among the samples and the classifier might only need a small amount of low informative samples to learn the patterns. Random sampling may overlook sample informativeness, which consumes human labelling costs and contributes less to training the classifiers. As an alternative, researchers suggest employing statistical sampling methods of Active Learning (AL) to identify the informative samples for training the classifiers. However, the use of AL methods in educational DA classification tasks is under-explored. In this paper, we examine the informativeness of annotated sentence samples. Then, the study investigates how the AL methods can select informative samples to support DA classifiers in the AL sampling process. The results reveal that most annotated sentences present low informativeness in the training dataset and the patterns of these sentences can be easily captured by the DA classifier. We also demonstrate how AL methods can reduce the cost of manual annotation in the AL sampling process.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
Due to the space limit, we only present a part of the analysis results, and the full results can be accessed at https://github.com/jionghaolin/INFO.
- 3.
Due to the space limit, we documented our full results in a digital appendix, which is accessible via https://github.com/jionghaolin/INFO.
References
Boyer, K., Ha, E.Y., Phillips, R., Wallis, M., Vouk, M., Lester, J.: Dialogue act modeling in a complex task-oriented domain. In: Proceedings of the SIGDIAL 2010 Conference, pp. 297–305 (2010)
Clark, K., Luong, M.T., Le, Q.V., Manning, C.D.: Electra: pre-training text encoders as discriminators rather than generators. In: ICLR (2019)
D’Mello, S., Olney, A., Person, N.: Mining collaborative patterns in tutorial dialogues. J. Educ. Data Min. 2(1), 1–37 (2010)
Du, B., et al.: Exploring representativeness and informativeness for active learning. IEEE Trans. Cybern. 47(1), 14–26 (2015)
Du Boulay, B., Luckin, R.: Modelling human teaching tactics and strategies for tutoring systems: 14 years on. IJAIED 26(1), 393–404 (2016)
Ezen-Can, A., Boyer, K.E.: Understanding student language: an unsupervised dialogue act classification approach. JEDM 7(1), 51–78 (2015)
Ezen-Can, A., Grafsgaard, J.F., Lester, J.C., Boyer, K.E.: Classifying student dialogue acts with multimodal learning analytics. In: Proceedings of the Fifth LAK, pp. 280–289 (2015)
Hastings, P., Hughes, S., Britt, M.A.: Active learning for improving machine learning of student explanatory essays. In: Penstein Rosé, C., et al. (eds.) AIED 2018. LNCS (LNAI), vol. 10947, pp. 140–153. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93843-1_11
Hennessy, S., et al.: Developing a coding scheme for analysing classroom dialogue across educational contexts. Learn. Cult. Soc. Interact. 9, 16–44 (2016)
Karamcheti, S., Krishna, R., Fei-Fei, L., Manning, C.D.: Mind your outliers! Investigating the negative impact of outliers on active learning for visual question answering. In: Proceedings of the 59th ACL, pp. 7265–7281 (2021)
Karumbaiah, S., Lan, A., Nagpal, S., Baker, R.S., Botelho, A., Heffernan, N.: Using past data to warm start active machine learning: does context matter? In: LAK21: 11th LAK, pp. 151–160 (2021)
Lin, J., et al.: Is it a good move? Mining effective tutoring strategies from human-human tutorial dialogues. Futur. Gener. Comput. Syst. 127, 194–207 (2022)
Lin, J., et al.: Enhancing educational dialogue act classification with discourse context and sample informativeness. IEEE TLT (in press)
Nye, B.D., Graesser, A.C., Hu, X.: Autotutor and family: a review of 17 years of natural language tutoring. IJAIED 24(4), 427–469 (2014)
Nye, B.D., Morrison, D.M., Samei, B.: Automated session-quality assessment for human tutoring based on expert ratings of tutoring success. Int. Educ. Data Min. Soc. (2015)
Rus, V., Maharjan, N., Banjade, R.: Dialogue act classification in human-to-human tutorial dialogues. In: Innovations in Smart Learning. LNET, pp. 183–186. Springer, Singapore (2017). https://doi.org/10.1007/978-981-10-2419-1_25
Rus, V., et al.: An analysis of human tutors’ actions in tutorial dialogues. In: The Thirtieth International Flairs Conference (2017)
Samei, B., Li, H., Keshtkar, F., Rus, V., Graesser, A.C.: Context-based speech act classification in intelligent tutoring systems. In: Trausan-Matu, S., Boyer, K.E., Crosby, M., Panourgia, K. (eds.) ITS 2014. LNCS, vol. 8474, pp. 236–241. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-07221-0_28
Settles, B.: Active Learning. Synthesis Digital Library of Engineering and Computer Science. Morgan & Claypool, San Rafael (2012)
Sha, L., Li, Y., Gasevic, D., Chen, G.: Bigger data or fairer data? Augmenting BERT via active sampling for educational text classification. In: Proceedings of the 29th COLING, pp. 1275–1285 (2022)
Swayamdipta, S., et al.: Dataset cartography: mapping and diagnosing datasets with training dynamics. In: Proceedings of EMNLP. ACL, Online (2020)
Tan, W., Du, L., Buntine, W.: Diversity enhanced active learning with strictly proper scoring rules. In: NeurIPS, vol. 34 (2021)
Vail, A.K., Grafsgaard, J.F., Boyer, K.E., Wiebe, E.N., Lester, J.C.: Predicting learning from student affective response to tutor questions. In: Micarelli, A., Stamper, J., Panourgia, K. (eds.) ITS 2016. LNCS, vol. 9684, pp. 154–164. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-39583-8_15
Vail, A.K., Boyer, K.E.: Identifying effective moves in tutoring: on the refinement of dialogue act annotation schemes. In: Trausan-Matu, S., Boyer, K.E., Crosby, M., Panourgia, K. (eds.) ITS 2014. LNCS, vol. 8474, pp. 199–209. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-07221-0_24
Yang, Y., Loog, M.: Active learning using uncertainty information. In: Proceedings of 23rd International Conference on Pattern Recognition, pp. 2646–2651 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Tan, W. et al. (2023). Does Informativeness Matter? Active Learning for Educational Dialogue Act Classification. In: Wang, N., Rebolledo-Mendez, G., Matsuda, N., Santos, O.C., Dimitrova, V. (eds) Artificial Intelligence in Education. AIED 2023. Lecture Notes in Computer Science(), vol 13916. Springer, Cham. https://doi.org/10.1007/978-3-031-36272-9_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-36272-9_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-36271-2
Online ISBN: 978-3-031-36272-9
eBook Packages: Computer ScienceComputer Science (R0)