research-article

Exploring Document Retrieval Features Associated with Improved Short- and Long-term Vocabulary Learning Outcomes

Authors:

Kevyn Collins-ThompsonAuthors Info & Claims

CHIIR '18: Proceedings of the 2018 Conference on Human Information Interaction & Retrieval

Pages 191 - 200

https://doi.org/10.1145/3176349.3176397

Published: 01 March 2018 Publication History

Abstract

A growing body of information retrieval research has studied the potential of search engines as effective, scalable platforms for self-directed learning. Towards this goal, we explore document representations for retrieval that include features associated with effective learning outcomes. While prior studies have investigated different retrieval models designed for teaching, this study is the first to investigate how document-level features are associated with actual learning outcomes when users get results from a personalized learning-oriented retrieval algorithm. We also conduct what is, to our knowledge, the first crowdsourced longitudinal study of long-term learning retention, in which we gave a subset of users who participated in an initial learning and assessment study a delayed post-test approximately nine months later. With this data, we were able to analyze how the three retrieval conditions in the original study were associated with changes in long-term vocabulary knowledge. We found that while users who read the documents in the personalized retrieval condition had immediate learning gains comparable to the other two conditions, they had better long-term retention of more difficult vocabulary.

References

[1]

Mustafa Abualsaud . 2017. Learning Factors and Determining Document-level Satisfaction In Search-as-Learning. Master's thesis. bibinfoschoolUniversity of Waterloo.

[2]

Peter Bailey, Liwei Chen, Scott Grosenick, Li Jiang, Yan Li, Paul Reinholdtsen, Charles Salada, Haidong Wang, and Sandy Wong . 2012. User task understanding: a web search engine perspective NII Shonan Meeting on Whole-Session Evaluation of Interactive Information Retrieval Systems, Kanagawa, Japan.

[3]

Elizabeth Ligon Bjork, Jeri L Little, and Benjamin C Storm . 2014. Multiple-choice testing as a desirable difficulty in the classroom. Journal of Applied Research in Memory and Cognition, Vol. 3, 3 (2014), 165--170.

[4]

Bertram C Brookes . 1980. The foundations of information science Part I. Philosophical aspects. Journal of Information Science Vol. 2, 3--4 (1980), 125--133.

Digital Library

[5]

Kevyn Collins-Thompson and Jamie Callan . 2004. Information Retrieval for Language Tutoring: An Overview of the REAP Project Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '04). ACM, New York, NY, USA, 544--545.

Digital Library

[6]

Kevyn Collins-Thompson, Soo Young Rieh, Carl C. Haynes, and Rohail Syed . 2016. Assessing Learning Outcomes in Web Search: A Comparison of Tasks and Query Strategies Proceedings of the 2016 ACM on Conference on Human Information Interaction and Retrieval (CHIIR '16). ACM, New York, NY, USA, 163--172.

Digital Library

[7]

Cathy De Rosa . 2006. College students' perceptions of libraries and information resources: A report to the OCLC membership. OCLC.

[8]

Diana DeStefano and Jo-Anne LeFevre . 2007. Cognitive load in hypertext reading: A review. Computers in Human Behavior Vol. 23, 3 (2007), 1616--1641.

Digital Library

[9]

Geoffrey B Duggan and Stephen J Payne . 2008. Knowledge in the head and on the web: Using topic expertise to aid search Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 39--48.

Digital Library

[10]

Carsten Eickhoff, Jaime Teevan, Ryen White, and Susan Dumais . 2014. Lessons from the Journey: A Query Log Analysis of Within-session Learning Proceedings of the 7th ACM International Conference on Web Search and Data Mining (WSDM '14). ACM, New York, NY, USA, 223--232.

Digital Library

[11]

Luanne Freund, Rick Kopak, and Heather O'Brien . 2016. The effects of textual environment on reading comprehension: Implications for searching as learning. Journal of Information Science Vol. 42, 1 (2016), 79--93.

Digital Library

[12]

Bernard J Jansen, Danielle Booth, and Brian Smith . 2009. Using the taxonomy of cognitive learning to model online searching. Information Processing & Management Vol. 45, 6 (2009), 643--663.

Digital Library

[13]

Terry Judd and Gregor Kennedy . 2010. A five-year study of on-campus Internet use by undergraduate biomedical students. Computers & Education Vol. 55, 4 (2010), 1564--1571.

Digital Library

[14]

Jin Young Kim, Kevyn Collins-Thompson, Paul N. Bennett, and Susan T. Dumais . 2012. Characterizing Web Content, User Interests, and Search Behavior by Reading Level and Topic Proceedings of the Fifth ACM International Conference on Web Search and Data Mining (WSDM '12). ACM, New York, NY, USA, 213--222.

Digital Library

[15]

Victor Kuperman, Hans Stadthagen-Gonzalez, and Marc Brysbaert . 2012. Age-of-acquisition ratings for 30,000 English words. Behavior Research Methods Vol. 44, 4 (2012), 978--990.

[16]

Richard E Mayer . 1997. Multimedia learning: Are we asking the right questions? Educational Psychologist Vol. 32, 1 (1997), 1--19.

[17]

Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean . 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013).

[18]

Soo Young Rieh, Kevyn Collins-Thompson, Preben Hansen, and Hye-Jung Lee . 2016. Towards searching as a learning process: A review of current perspectives and future directions. Journal of Information Science Vol. 42, 1 (2016), 19--34.

Digital Library

[19]

Rohail Syed and Kevyn Collins-Thompson . 2017 a. Optimizing search results for human learning goals. Information Retrieval Journal (2017), 1--18.

Digital Library

[20]

Rohail Syed and Kevyn Collins-Thompson . 2017 b. Retrieval algorithms optimized for human learning. Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 555--564.

Digital Library

[21]

Manisha Verma, Emine Yilmaz, and Nick Craswell . 2016. On Obtaining Effort Based Judgements for Information Retrieval Proceedings of the Ninth ACM International Conference on Web Search and Data Mining. ACM, 277--286.

Digital Library

[22]

Ryen W White, Susan T Dumais, and Jaime Teevan . 2009. Characterizing the influence of domain expertise on web search behavior Proceedings of the Second ACM International Conference on Web Search and Data Mining. ACM, 132--141.

Digital Library

[23]

Barbara M Wildemuth . 2004. The effects of domain knowledge on search tactic formulation. Journal of the American Society for Information Science and Technology, Vol. 55, 3 (2004), 246--258.

Digital Library

[24]

Wan-Ching Wu, Diane Kelly, Ashlee Edwards, and Jaime Arguello . 2012. Grannies, tanning beds, tattoos and NASCAR: Evaluation of search tasks with varying levels of cognitive complexity. In Proceedings of the 4th Information Interaction in Context Symposium. ACM, 254--257.

Digital Library

[25]

Joerg Zumbach and Maryam Mohraz . 2008. Cognitive load in hypermedia reading comprehension: Influence of text type and linearity. Computers in Human Behavior Vol. 24, 3 (2008), 875--887.

Digital Library

Cited By

El Zein DDa Costa Pereira C(2023)The Evolution of User Knowledge during Search-as-Learning Sessions: A Benchmark and BaselineProceedings of the 2023 Conference on Human Information Interaction and Retrieval10.1145/3576840.3578273(454-458)Online publication date: 19-Mar-2023
https://dl.acm.org/doi/10.1145/3576840.3578273
Liu JLiu J(2023)From Rational Agent to Human with Bounded RationalityA Behavioral Economics Approach to Interactive Information Retrieval10.1007/978-3-031-23229-9_3(65-89)Online publication date: 18-Feb-2023
https://doi.org/10.1007/978-3-031-23229-9_3
El Zein Dda Costa Pereira CBellogin ABoratto LSantos OArdissono LKnijnenburg B(2022)User’s Knowledge and Information Needs in Information Retrieval EvaluationProceedings of the 30th ACM Conference on User Modeling, Adaptation and Personalization10.1145/3503252.3531325(170-178)Online publication date: 4-Jul-2022
https://dl.acm.org/doi/10.1145/3503252.3531325
Show More Cited By

Index Terms

Exploring Document Retrieval Features Associated with Improved Short- and Long-term Vocabulary Learning Outcomes

Recommendations

A novel long-term learning algorithm for relevance feedback in content-based image retrieval

Relevance feedback has proven to be a powerful tool to bridge the semantic gap between low-level features and high-level human concepts in content-based image retrieval (CBIR). However, traditional short-term relevance feedback technologies are confined ...
A Relevance Feedback System for CBIR with Long-Term Learning
MINES '10: Proceedings of the 2010 International Conference on Multimedia Information Networking and Security

Relevance feedback has been developed to improve retrieval performance effectively in Content Based Image Retrieval (CBIR). This paper introduces a relevance feedback system for CBIR with both short-term relevance feedback and long-term learning. In ...
Long-term learning in content-based image retrieval
Multimedia Information Retrieval

In content-based image retrieval, relevance feedback is an interactive process, which builds a bridge to connect users with the search engine. It leads to much improved retrieval performance by updating the query and the similarity measure according to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHIIR '18: Proceedings of the 2018 Conference on Human Information Interaction & Retrieval

March 2018

402 pages

ISBN:9781450349253

DOI:10.1145/3176349

General Chairs:
Chirag Shah
Rutgers University, USA
,
Nicholas J. Belkin
Rutgers University, USA
,
Program Chairs:
Katriina Byström
Oslo Metropolitan University, Norway
,
Jeff Huang
Brown University, USA
,
Falk Scholer
RMIT University, Australia

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

In-Cooperation

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 March 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Best Student Paper

Author Tags

Qualifiers

Research-article

Funding Sources

Institute of Education Sciences

Conference

CHIIR '18

Sponsor:

SIGIR

CHIIR '18: Conference on Human Information Interaction and Retrieval

March 11 - 15, 2018

NJ, New Brunswick, USA

Acceptance Rates

CHIIR '18 Paper Acceptance Rate 22 of 57 submissions, 39%;

Overall Acceptance Rate 55 of 163 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
296
Total Downloads

Downloads (Last 12 months)13
Downloads (Last 6 weeks)1

Reflects downloads up to 13 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

El Zein DDa Costa Pereira C(2023)The Evolution of User Knowledge during Search-as-Learning Sessions: A Benchmark and BaselineProceedings of the 2023 Conference on Human Information Interaction and Retrieval10.1145/3576840.3578273(454-458)Online publication date: 19-Mar-2023
https://dl.acm.org/doi/10.1145/3576840.3578273
Liu JLiu J(2023)From Rational Agent to Human with Bounded RationalityA Behavioral Economics Approach to Interactive Information Retrieval10.1007/978-3-031-23229-9_3(65-89)Online publication date: 18-Feb-2023
https://doi.org/10.1007/978-3-031-23229-9_3
El Zein Dda Costa Pereira CBellogin ABoratto LSantos OArdissono LKnijnenburg B(2022)User’s Knowledge and Information Needs in Information Retrieval EvaluationProceedings of the 30th ACM Conference on User Modeling, Adaptation and Personalization10.1145/3503252.3531325(170-178)Online publication date: 4-Jul-2022
https://dl.acm.org/doi/10.1145/3503252.3531325
Arakawa RYakura HKobayashi S(2022)VocabEncounter: NMT-powered Vocabulary Learning by Presenting Computer-Generated Usages of Foreign Words into Users’ Daily LivesProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501839(1-21)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3501839
Câmara AMaxwell DHauff C(2022)Searching, Learning, and Subtopic Ordering: A Simulation-Based AnalysisAdvances in Information Retrieval10.1007/978-3-030-99736-6_10(142-156)Online publication date: 5-Apr-2022
https://doi.org/10.1007/978-3-030-99736-6_10
Roy NTorre MGadiraju UMaxwell DHauff CScholer FThomas PElsweiler DJoho HKando NSmith C(2021)Note the HighlightProceedings of the 2021 Conference on Human Information Interaction and Retrieval10.1145/3406522.3446025(229-238)Online publication date: 14-Mar-2021
https://dl.acm.org/doi/10.1145/3406522.3446025
Yu RTang RRokicki MGadiraju UDietze S(2021)Topic-independent modeling of user knowledge in informational search sessionsInformation Retrieval Journal10.1007/s10791-021-09391-7Online publication date: 16-Mar-2021
https://doi.org/10.1007/s10791-021-09391-7
Otto CYu RPardi Gvon Hoyer JRokicki MHoppe AHoltz PKammerer YDietze SEwerth R(2021)Predicting Knowledge Gain During Web Search Based on Multimedia Resource ConsumptionArtificial Intelligence in Education10.1007/978-3-030-78292-4_26(318-330)Online publication date: 11-Jun-2021
https://doi.org/10.1007/978-3-030-78292-4_26
Roy NTorre MGadiraju UMaxwell DHauff C(2021)How Do Active Reading Strategies Affect Learning Outcomes in Web Search?Advances in Information Retrieval10.1007/978-3-030-72240-1_37(368-375)Online publication date: 30-Mar-2021
https://doi.org/10.1007/978-3-030-72240-1_37
Smith CRieh SAzzopardi LHalvey MRuthven IJoho HMurdock VQvarfordt P(2019)Knowledge-Context in Search SystemsProceedings of the 2019 Conference on Human Information Interaction and Retrieval10.1145/3295750.3298940(55-62)Online publication date: 8-Mar-2019
https://dl.acm.org/doi/10.1145/3295750.3298940

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents