Abstract
Conversational interfaces (also called chatbots) have recently disrupted the Internet and opened up endless opportunities for assessment and learning. Formative feedback that provides learners with practical instructions for improvement is one of the challenging tasks in self-assessment settings and self-directed learning. This becomes even more challenging if a user’s personal information such as learning history and previous achievements cannot be exploited for data protection reasons or are simply not available. This study seeks to explore the opportunities of providing formative feedback in chatbot-based self-assessment. Two main challenges were faced: the limitations of the messenger as an interface that restricts visual representation of the quiz questions, and zero information about the user to generate adaptive feedback. Two types of feedback were investigated regarding their formative effect: immediate feedback, which was given after answering a question, and cumulative feedback detailing strengths and weaknesses of the user in each of the topics covered along with the directives for improvement. A chatbot called SQL Quizbot was deployed on Facebook Messenger for the purposes of this study (Try out the prototype at https://www.messenger.com/t/2076690849324267). A survey conducted to disclose users’ perception of the feedback reveals that more than 80% of the users find immediate feedback helpful. Overall this study shows that chatbots have a great potential as an aiding tool for e-learning systems to include an interactive component into feedback in order to increase user motivation and retention.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
http://uk.businessinsider.com/the-messaging-app-report-2015-11?r=US&IR=T, last checked 22.11.2018.
- 2.
- 3.
- 4.
References
Muirhead, B., Juwah, C.: Interactivity in computer-mediated college and university education: a recent review of the literature. J. Educ. Technol. Soc. 7(1), 12–20 (2004)
Zheng, S., Rosson, M.B., Shih, P.C., Carroll, J.M.: Understanding student motivation, behaviors and perceptions in MOOCs. In: Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & #38; Social Computing, CSCW 2015, pp. 1882–1895. ACM, New York (2015)
Weizenbaum, J.: ELIZA - a computer program for the study of natural language communication between man and machine. Commun. ACM 9, 36–45 (1966)
Shute, V.J.: Focus on formative feedback. Rev. Educ. Res. 78(1), 153–189 (2008)
Hattie, J., Timperley, H.: The power of feedback. Rev. Educ. Res. 77(1), 81–112 (2007)
Lipowsky, F.: Unterricht. In: Wild, E., Möller, J. (eds.) Pädagogische Psychologie. SLB, pp. 69–105. Springer, Heidelberg (2015). https://doi.org/10.1007/978-3-642-41291-2_4
Wiliam, D.: Embedded Formative Assessment. Solution Tree Press, Bloomington (2011)
Höhn, S., Ras, E.: Designing formative and adaptive feedback using incremental user models. In: Chiu, D.K.W., Marenzi, I., Nanni, U., Spaniol, M., Temperini, M. (eds.) ICWL 2016. LNCS, vol. 10013, pp. 172–177. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-47440-3_19
Denton, P., Madden, J., Roberts, M., Rowe, P.: Students’ response to traditional and computer-assisted formative feedback: a comparative case study. Br. J. Educ. Technol. 39(3), 486–500 (2008)
Black, P., Wiliam, D.: Developing the theory of formative assessment. Educ. Assess. Eval. Account. (Former.: J. Pers. Eval. Educ.) 21(1), 5 (2009)
Wiggins, G.: Seven keys to effective feedback. Educ. Leadersh. 70(1), 10–16 (2012)
Espasa, A., Guasch, T., Mayordomo, R., Martínez-Melo, M., Carless, D.: A dialogic feedback index measuring key aspects of feedback processes in online learning environments. High. Educ. Res. Dev. 37(3), 499–513 (2018)
Narciss, S.: Designing and evaluating tutoring feedback strategies for digital learning. Digit. Educ. Rev. 23, 7–26 (2013)
Butler, A.C., Roediger, H.L.: Feedback enhances the positive effects and reduces the negative effects of multiple-choice testing. Mem. Cogn. 36(3), 604–616 (2008)
Bälter, O., Enström, E., Klingenberg, B.: The effect of short formative diagnostic web quizzes with minimal feedback. Comput. Educ. 60(1), 234–242 (2013)
De Klerk, S., Veldkamp, B.P., Eggen, T.: The psychometric evaluation of a summative multimedia-based performance assessment. In: Ras, E., Joosten-ten Brinke, D. (eds.) CAA 2015. CCIS, vol. 571, pp. 1–11. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-27704-2_1
Novacek, P.: Confidence-based assessments within an adult learning environment. In: International Association for Development of the Information Society, pp. 403–406 (2013)
Hench, T.L.: Using confidence as feedback in multi-sized learning environments. In: Kalz, M., Ras, E. (eds.) CAA 2014. CCIS, vol. 439, pp. 88–99. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-08657-6_9
Lundeberg, M.A., Fox, P.W., Punćcohaí, J.: Highly confident but wrong: gender differences and similarities in confidence judgments. J. Educ. Psychol. 86(1), 114 (1994)
Jonsson, A.C., Allwood, C.M.: Stability and variability in the realism of confidence judgments over time, content domain, and gender. Pers. Individ. Differ. 34(4), 559–574 (2003)
Burns, K.M., Burns, N.R., Ward, L.: Confidence–more a personality or ability trait? It depends on how it is measured: a comparison of young and older adults. Front. Psychol. 7, 518 (2016)
West, R.F., Stanovich, K.E.: The domain specificity and generality of overconfidence: individual differences in performance estimation bias. Psychon. Bull. Rev. 4(3), 387–392 (1997)
Gardner-Medwin, A.: Confidence assessment in the teaching of basic science. ALT-J 3(1), 80–85 (1995)
Gardner-Medwin, A.: 12 confidence-based marking. In: Innovative Assessment in Higher Education, p. 141 (2006)
Ericsson, K.A., Krampe, R.T., Tesch-Römer, C.: The role of deliberate practice in the acquisition of expert performance. Psychol. Rev. 100(3), 363–406 (1993)
Christodoulou, D., Wiliam, D.: Making Good Progress?: The Future of Assessment for Learning. Oxford University Press, Oxford (2017)
Roediger, H.L., Karpicke, J.D.: The power of testing memory: basic research and implications for educational practice. Perspect. Psychol. Sci. 1(3), 181–210 (2006). PMID: 26151629
Petersen, K.A.: Implicit corrective feedback in computer-guided interaction: does mode matter? Ph.D. thesis, Georgetown University (2010)
Wilske, S.: Form and meaning in dialog-based computer-assisted language learning. Ph.D. thesis, University of Saarland (2014)
Kerly, A., Hall, P., Bull, S.: Bringing chatbots into education: towards natural language negotiation of open learner models. Knowl.-Based Syst. 20(2), 177–185 (2007)
Kane, D.A.: The role of chatbots in teaching and learning. In: E-Learning and the Academic Library: Essays on Innovative Initiatives, UC Irvine, pp. 1–26 (2016)
Soliman, M., Guetl, C.: Intelligent pedagogical agents in immersive virtual learning environments: a review. In: MIPRO 2010 Proceedings of the 33rd International Convention, pp. 827–832. IEEE (2010)
MacTear, M., Callejas, Z., Griol, D.: The Conversational Interface: Talking to Smart Devices. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-32967-3
DeSmedt, W.H.: Herr Kommissar: an ICALL conversation simulator for intermediate German. In: Holland, V.M., Sams, M.R., Kaplan, J.D. (eds.) Intelligent Language Tutors: Theory Shaping Technology. Routledge, New York (1995)
Lu, C.-H., Chiou, G.-F., Day, M.-Y., Ong, C.-S., Hsu, W.-L.: Using instant messaging to provide an intelligent learning environment. In: Ikeda, M., Ashley, K.D., Chan, T.-W. (eds.) ITS 2006. LNCS, vol. 4053, pp. 575–583. Springer, Heidelberg (2006). https://doi.org/10.1007/11774303_57
Jia, J.: CSIEC: a computer assisted english learning chatbot based on textual knowledge and reasoning. Knowl.-Based Syst. 22(4), 249–255 (2009)
Höhn, S.: A data-driven model of explanations for a chatbot that helps to practice conversation in a foreign language. In: Proceedings of SIGDIAL 2017 Conference. ACM (2017)
Lyster, R., Ranta, L.: Corrective feedback and learner uptake. Stud. Second Lang. Acquis. 19(01), 37–66 (1997)
Lyster, R., Saito, K., Sato, M.: Oral corrective feedback in second language classrooms. Lang. Teach. 46, 1–40 (2013)
Amaral, L.A., Meurers, D.: On using intelligent computer-assisted language learning in real-life foreign language teaching and learning. ReCALL 23(01), 4–24 (2011)
Gross, S., Pinkwart, N.: Towards an integrative learning environment for Java programming. In: 2015 IEEE 15th International Conference on Advanced Learning Technologies (ICALT), pp. 24–28, July 2015
Perikos, I., Grivokostopoulou, F., Hatzilygeroudis, I.: Assistance and feedback mechanism in an intelligent tutoring system for teaching conversion of natural language into logic. Int. J. Artif. Intell. Educ. 27(3), 475–514 (2017)
Ryan, T., Henderson, M.: Feeling feedback: students’ emotional responses to educator feedback. Assess. Eval. High. Educ. 43(6), 880–892 (2018)
Palminteri, S., Khamassi, M., Joffily, M., Coricelli, G.: Contextual modulation of value signals in reward and punishment learning. Nat. Commun. 6, 8096 (2015). https://doi.org/10.1038/ncomms9096
Ras, E., Baudet, A., Foulonneau, M.: A hybrid engineering process for semi-automatic item generation. In: Joosten-ten Brinke, D., Laanpere, M. (eds.) TEA 2016. CCIS, vol. 653, pp. 105–116. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-57744-9_10
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Vijayakumar, B., Höhn, S., Schommer, C. (2019). Quizbot: Exploring Formative Feedback with Conversational Interfaces. In: Draaijer, S., Joosten-ten Brinke, D., Ras, E. (eds) Technology Enhanced Assessment. TEA 2018. Communications in Computer and Information Science, vol 1014. Springer, Cham. https://doi.org/10.1007/978-3-030-25264-9_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-25264-9_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-25263-2
Online ISBN: 978-3-030-25264-9
eBook Packages: Computer ScienceComputer Science (R0)