Abstract
A computer-adaptive test (CAT) is a software application that makes use of Item Response Theory (IRT) to create a test that is tailored to individual learners. The CAT prototype introduced here comprised a graphical user interface, a question database and an adaptive algorithm based on the Three-Parameter Logistic Model from IRT. A sample of 113 Computer Science undergraduate students participated in a session of assessment within the Human-Computer Interaction subject domain using our CAT prototype. At the end of the assessment session, participants were asked to rate the level of difficulty of the overall test from 1 (very easy) to 5 (very difficult). The perceived level of difficulty of the test and the CAT scores obtained by this group of learners were subjected to a Spearman’s rank order correlation. Findings from this statistical analysis suggest that the CAT prototype was effective in tailoring the assessment to each individual learner’s proficiency level.
Chapter PDF
Similar content being viewed by others
Keywords
- Item Response Theory
- Proficiency Level
- Educational Software
- Assessment Session
- Computer Adaptive Testing
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Barker, T., Lilley, M.: Are Individual Learners Disadvantaged by the Use of Computer-Adaptive Testing? In: Proceedings of the 8th Learning Styles Conference, University of Hull. European Learning Styles Information Network (ELSIN), pp. 30–39 (2003)
Brusilovsky, P.: Knowledge Tree: A Distributed Architecture for Adaptive E-Learning. In: Proceedings of the 13th World Wide Web Conference, New York, USA, May 17-22, pp. 104–113 (2004)
Conejo, R., Millán, E., Pérez-de-la-Cruz, J.L., Trella, M.: An Empirical Approach to On-Line Learning in SIETTE. In: Gauthier, G., VanLehn, K., Frasson, C. (eds.) ITS 2000. LNCS, vol. 1839, pp. 605–614. Springer, Heidelberg (2000)
Fernandez, G.: Cognitive Scaffolding for a Web-Based Adaptive Learning Environment. In: Zhou, W., Nicholson, P., Corbitt, B., Fong, J. (eds.) ICWL 2003. LNCS, vol. 2783, pp. 12–20. Springer, Heidelberg (2003)
Jettmar, E., Nass, C.: Adaptive testing: effects on user performance. In: Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, Minneapolis, Minnesota, USA, pp. 129–134 (2002)
Lilley, M., Barker, T., Britton, C.: The development and evaluation of a software prototype for computer adaptive testing. Computers & Education Journal 43(1-2), 109–122 (2004)
Lord, F.M.: Applications of Item Response Theory to practical testing problems. Lawrence Erlbaum Associates, New Jersey (1980)
Wainer, H.: Computerized Adaptive Testing (A Primer), 2nd edn. Lawrence Erlbaum Associates, New Jersey (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 IFIP International Federation for Information Processing
About this paper
Cite this paper
Lilley, M., Barker, T., Britton, C. (2005). Learners’ Perceived Level of Difficulty of a Computer-Adaptive Test: A Case Study. In: Costabile, M.F., Paternò, F. (eds) Human-Computer Interaction - INTERACT 2005. INTERACT 2005. Lecture Notes in Computer Science, vol 3585. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11555261_96
Download citation
DOI: https://doi.org/10.1007/11555261_96
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-28943-2
Online ISBN: 978-3-540-31722-7
eBook Packages: Computer ScienceComputer Science (R0)