Abstract
Current work in learner evaluation of Intelligent Tutoring Systems (ITSs), is moving towards open-ended educational content diagnosis. One of the main difficulties of this approach is to be able to automatically understand natural language. Our work is directed to produce automatic evaluation of learner summaries in Basque. Therefore, in addition to language comprehension, difficulties emerge from Basque morphology itself. In this work, Latent Semantic Analysis (LSA) is used to model comprehension in a language in which lemmatization has shown to be highly significant. This paper tests the influence of corpus lemmatization while performing automatic comprehension and coherence grading. Summaries graded by human judges in coherence and comprehension, have been tested against LSA based measures from source lemmatized and non-lemmatized corpora. After lemmatization, the amount of LSA known single terms was reduced in a 56% of its original number. As a result, LSA grades almost match human measures, producing no significant differences between the lemmatized and non-lemmatized approaches.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Kintsch, W., Patel, V.L., Ericsson, K.A.: The role of long-term working memory in text comprehension. Psychologia 42, 186–198 (1999)
Barlett, F.C.: Remembering; a Studty in Experimental and Social Psychology. Cambridge University Press, Cambridge (1932)
Garner, R.: Efficient Text Summarization. Costs and Benefits. Journal of Educational Research 75(5), 275–279 (1982)
Zipitria, I., Elorriaga, J.A., Arruarte, A., de Ilarraza, A.D.: From Human to Automatic Summary Evaluation. In: Lester, J.C., Vicari, R.M., Paraguaçu, F. (eds.) ITS 2004. LNCS, vol. 3220, pp. 432–442. Springer, Heidelberg (2004)
Landauer, T.K., Dumais, S.T.: A solution to Plato’s problem: The Latent Semantic Analysis theory of acquisition, induction, and representation of knowledge. Psychological Review 104, 211–240 (1997)
Deerwester, S., Dumais, S.T., Furnas, G.W., Landauer, T.K., Harshman, R.: Indexing by Latent Semantic Analysis. Journal of the American Society of Information Science (1990)
Landauer, T.K., Foltz, P., Laham, D.: Introduction to Latent Semantic Analysis. Discourse Processes 25, 259–284 (1998)
Foltz, P.W., Kintsch, W., Landauer, T.K.: The Measurement of Textual Coherence with Latent Semantic Analysis. Discourse Processes 25, 285–307 (1998)
Wolfe, M.B.W., Schreiner, M.E., Rehder, B., Laham, D., Foltz, P.W., Kintsch, W., Lan-dauer, T.K.: Learning from text:Matching readers and texts by Latent Semantic Analysis. Discourse Processes 25, 309–336 (1998)
Graesser, A.C., Person, N.K., Harter, D.: Teaching tactics and dialog in Autotutor. International Journal of Artificial Intelligence in Education 12, 257–279 (2001)
Wiemer-Hastings, P., Graesser, A.: Select-a-Kibitzer: A computer tool that gives meaningful feedback on student compositions. Interactive Learning Environments 8(2), 149–169 (2000)
Wade-Stein, D., Kintsch, E.: Summary Street: Interactive Computer Support for Writing. Cognition and Instruction 22(3), 333–362 (2004)
Miller, T.: Essay assessment with latent semantic analysis. Journal of Educational Computing Research 28 (2003)
Ventura, M.J., Franchescetti, D.R., Pennumatsa, P., Graesser, A.C., Hu, G.T.J.X., Cai, Z., Group, t.T.R.: Combining Computational Models of Short Essay Grading for Conceptual Physics Problems. In: Lester, J.C., Vicari, R.M., Paraguaçu, F. (eds.) ITS 2004. LNCS, vol. 3220, pp. 423–431. Springer, Heidelberg (2004)
Tomasello, M.: Constructing a Language: A Usage-Based Theory of Language Acquisition. Harvard University Press, Cambridge (2003)
Palolahti, M., Leino, S., Jokela, M., Kopra, K., Paavilainen, P.: Event-related potentials suggest early interaction between syntax and semantics during on-line sentence comprehension. Neuroscience Letters 384(3), 222 (2005)
Hagoort, P.: Interplay between Syntax and Semantics during Sentence Comprehension: ERP Effects of Combining Syntactic and Semantic Violations. Journal of Cognitive Neuroscience 15(6), 883–899 (2003)
Landauer, T.K., Laham, D., Rehder, B., Schreiner, M.E.: How well can passage meaning be derived without using word order? A comparison of Latent Semantic Analysis and humans. In: 19th Annual Meeting of the Cognitive Science Society. Erlbaum, Mahwah (1997)
Wiemer-Hastings, P., Zipitria, I.: Rules for Syntax, Vectors for Semantics. In: Proceedings of the 23rd Annual Conference of the Cognitive Science Society. Erlbaum, Mahwah (2001)
Serafin, R., Eugenio, B.D.: FLSA: Extending Latent Semantic Analysis with Features for Dialogue Act Classification. In: Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics, Barcelona, Spain (2004)
Kanejiya, D., Kamar, A., Prasad, S.: Automatic Evaluation of Students’ Answers using Syntactically Enhanced LSA. In: Proceedings of the HLT-NAACL 2003 Workshop on Building Educational Applications Using Natural Language Processing (2003)
Olde, B.A., Franceschetti, D.R., Karnavat, A., Graesser, A.C., TRG.: The right stuff: Do you need to sanitize your corpus when using latent semantic analysis? In: 24rd Annual Conference of the Cognitive Science Society. Erlbaum, Mahwah (2002)
Landauer, T.K., Littman, M.L.: A statistical method for language-independent representation of the topical content of text segments. In: Proceedings of the Sixth Annual Conference of the UW Centre for the New Oxford English Dictionary and Text Research (1990)
Aduriz, I., Aranzabe, M.J., Arriola, J.M., de Ilarraza, A.D., Gojenola, K., Oronoz, M., Uria, L.: A Cascaded Syntactic Analyser for Basque. In: Gelbukh, A. (ed.) CICLing 2004. LNCS, vol. 2945, pp. 124–134. Springer, Heidelberg (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zipitria, I., Arruarte, A., Elorriaga, J.A. (2006). Observing Lemmatization Effect in LSA Coherence and Comprehension Grading of Learner Summaries. In: Ikeda, M., Ashley, K.D., Chan, TW. (eds) Intelligent Tutoring Systems. ITS 2006. Lecture Notes in Computer Science, vol 4053. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11774303_59
Download citation
DOI: https://doi.org/10.1007/11774303_59
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-35159-7
Online ISBN: 978-3-540-35160-3
eBook Packages: Computer ScienceComputer Science (R0)