RLens: A Computer-aided Visualization System for Supporting Reflection on Language Learning under Distributed Tutorship
Authors: 
Meng Xia, Yankun Zhao, Jihyeong Hong, Mehmet Hamza Erol, Taewook Kim, Juho KimAuthors Info & Claims
Pages 60 - 71
Abstract
With the rise of the gig economy, online language tutoring platforms are becoming increasingly popular. These platforms provide temporary and flexible jobs for native speakers as tutors and allow language learners to have one-on-one speaking practices on demand, on which learners occasionally practice the language with different tutors. With such distributed tutorship, learners can hold flexible schedules and receive diverse feedback. However, learners face challenges in consistently tracking their learning progress because different tutors provide feedback from diverse standards and perspectives, and hardly refer to learners' previous experiences with other tutors. We present RLens, a visualization system for facilitating learners' learning progress reflection by grouping different tutors' feedback, tracking how each feedback type has been addressed across learning sessions, and visualizing the learning progress. We validate our design through a between-subjects study with 40 real-world learners. Results show that learners can successfully analyze their progress and common language issues under distributed tutorship with RLens, while most learners using the baseline interface had difficulty achieving reflection tasks. We further discuss design considerations of computer-aided systems for supporting learning under distributed tutorship.
Supplementary Material
Presentation video for the paper, RLens: A Computer-aided Visualization System for Supporting Reflection on Language Learning under Distributed Tutorship.
- Download
- 266.37 MB
References
[1]
Ali Alkhatib, Justin Cranshaw, and Andrés Monroy-Hernandez. 2018. Laying the Groundwork for a Worker-Centric Peer Economy. arXiv preprint arXiv:1807.08189 (2018).
[2]
Christopher Bryant, Mariano Felice, and Ted Briscoe. 2017. Automatic Annotation and Evaluation of Error Types for Grammatical Error Correction. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Vancouver, Canada, 793--805. https://doi.org/10.18653/v1/P17--1074
[3]
Yaohua Bu, Tianyi Ma, Weijun Li, Hang Zhou, Jia Jia, Shengqi Chen, Kaiyuan Xu, Dachuan Shi, Haozhe Wu, Zhihan Yang, et al. 2021. PTeacher: a Computer-Aided Personalized Pronunciation Training System with Exaggerated Audio-Visual Corrective Feedback. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1--14.
[4]
A.M. Colman. 2015. A Dictionary of Psychology. Oxford University Press.
[5]
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Unsupervised Cross-lingual Representation Learning at Scale. arXiv preprint arXiv:1911.02116 (2019).
[6]
Jill G De Villiers, Jill De Villiers, Peter A De Villiers, and Peter A DeVilliers. 1978. Language acquisition .Harvard University Press.
[7]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2--7, 2019, Volume 1 (Long and Short Papers), Jill Burstein, Christy Doran, and Thamar Solorio (Eds.). Association for Computational Linguistics, 4171--4186.
[8]
Berkeley J Dietvorst, Joseph P Simmons, and Cade Massey. 2015. Algorithm aversion: People erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology: General, Vol. 144, 1 (2015), 114.
[9]
Rod Ellis et al. 2003. Task-based language learning and teaching. Oxford University Press.
[10]
R. Ellis and Gary P. Barkhuizen. 2005. Analysing Learner Language .Oxford University Press.
[11]
K Anders Ericsson, Ralf T Krampe, and Clemens Tesch-Römer. 1993. The role of deliberate practice in the acquisition of expert performance. Psychological review, Vol. 100, 3 (1993), 363.
[12]
Hawa Fadhila. 2013. Errors In Speaking English Made By Students Ofenglish Department Of Muhammadiyah University Of Surakarta. Ph.D. Dissertation. Universitas Muhammadiyah Surakarta.
[13]
Mariano Felice, Christopher Bryant, and Ted Briscoe. 2016. Automatic Extraction of Learner Errors in ESL Sentences Using Linguistically Enhanced Alignments. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers. The COLING 2016 Organizing Committee, Osaka, Japan, 825--835.
[14]
Gerasimos Fergadiotis, Heather Harris Wright, and Samuel B Green. 2015. Psychometric evaluation of lexical diversity indices: Assessing length effects. Journal of Speech, Language, and Hearing Research, Vol. 58, 3 (2015), 840--852.
[15]
Pauline Foster, Alan Tonkyn, and Gillian Wigglesworth. 2000. Measuring spoken language: A unit for all reasons. Applied linguistics, Vol. 21, 3 (2000), 354--375.
[16]
Michael Gleicher, Danielle Albers, Rick Walker, Ilir Jusufi, Charles D Hansen, and Jonathan C Roberts. 2011. Visual comparison for information visualization. Information Visualization, Vol. 10, 4 (2011), 289--309.
[17]
Sten Govaerts, Katrien Verbert, Erik Duval, and Abelardo Pardo. 2012. The student activity meter for awareness and self-reflection. In CHI'12 Extended Abstracts on Human Factors in Computing Systems. 869--884.
[18]
Maarten Grootendorst. 2020. BERTopic: Leveraging BERT and c-TF-IDF to create easily interpretable topics. https://doi.org/10.5281/zenodo.4381785
[19]
Pam Harvey, Natalie Radomski, and Dennis O'Connor. 2013. Written feedback and continuity of learning in a geographically distributed medical education program. Medical teacher, Vol. 35, 12 (2013), 1009--1013.
[20]
John Hattie and Helen Timperley. 2007. The power of feedback. Review of educational research, Vol. 77, 1 (2007), 81--112.
[21]
Alex Housen and Folkert Kuiken. 2009. Complexity, accuracy, and fluency in second language acquisition. Applied linguistics, Vol. 30, 4 (2009), 461--473.
[22]
Diana L Hughes, LaRae McGillivray, Mark Schmidek, et al. 1997. Guide to narrative language: Procedures for assessment .Thinking Publications Eau Claire, WI.
[23]
Noriko Iwashita, Annie Brown, Tim McNamara, and Sally O'Hagan. 2008. Assessed levels of second language speaking proficiency: How distinct? Applied linguistics, Vol. 29, 1 (2008), 24--49.
[24]
Jelena Jovanović, Shane Dawson, Srećko Joksimović, and George Siemens. 2020. Supporting actionable intelligence: reframing the analysis of observed study strategies. In Proceedings of the Tenth International Conference on Learning Analytics & Knowledge. 161--170.
[25]
Igor Jugo, Bozidar Kovacic, and Vanja Slavuj. 2015. Integrating a Web-based ITS with DM tools for Providing Learning Path Optimization and Visual Analytics. In Proceedings of the 8th International Conference on Educational Data Mining, EDM 2015, Madrid, Spain, June 26--29, 2015. 574--575. http://www.educationaldatamining.org/EDM2015/proceedings/poster574--575.pdf
[26]
Jimin Kahng. 2014. Exploring utterance and cognitive fluency of L1 and L2 English speakers: Temporal measures and stimulated recall. Language Learning, Vol. 64, 4 (2014), 809--854.
[27]
Olga Kozar. 2015. Discursive practices of private online tutoring websites in Russia. Discourse: Studies in the cultural politics of education, Vol. 36, 3 (2015), 354--368.
[28]
Olga Kozar and Naomi Sweller. 2014. An exploratory study of demographics, goals and expectations of private online language learners in Russia. System, Vol. 45 (2014), 39--51.
[29]
Milos Krstajic, Enrico Bertini, and Daniel Keim. 2011. Cloudlines: Compact display of event episodes in multiple time-series. IEEE Transactions on Visualization and Computer Graphics, Vol. 17, 12 (2011), 2432--2439.
[30]
Yu-Ju Lan, Yao-Ting Sung, Chia-Chun Cheng, and Kuo-En Chang. 2015. Computer-supported cooperative prewriting for enhancing young EFL learners' writing performance. Language Learning & Technology, Vol. 19, 2 (2015), 134--155.
[31]
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. Albert: A lite bert for self-supervised learning of language representations. In ICLR .
[32]
Charlyn M. Laserna, Yi-Tai Seih, and James W. Pennebaker. 2014. Um. .. Who Like Says You Know: Filler Word Use as a Function of Age, Gender, and Personality. Journal of Language and Social Psychology, Vol. 33, 3 (2014), 328--338.
[33]
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv preprint arXiv:1907.11692 (2019).
[34]
Heng Luo, Anthony Robinson, and Jae-Young Park. 2014. Peer grading in a MOOC: Reliability, validity, and perceived effects. Online Learning Journal, Vol. 18, 2 (2014).
[35]
Roy Lyster and Leila Ranta. 1997. Corrective feedback and learner uptake: Negotiation of form in communicative classrooms. Studies in second language acquisition, Vol. 19, 1 (1997), 37--66.
[36]
Bill MacCartney. 2009. Natural language inference .Stanford University.
[37]
Philipp Mayring. 2015. Qualitative Content Analysis: Theoretical Background and Procedures .Springer Netherlands, Dordrecht, 365--380. https://doi.org/10.1007/978--94-017--9181--6_13
[38]
Riccardo Mazza and Vania Dimitrova. 2004. Visualising student tracking data to support instructors in web-based distance education. In Proceedings of the 13th International Conference on World Wide Web, WWW 2004, New York, NY, USA, May 17--20, 2004. 154--161. https://doi.org/10.1145/1013367.1013393
[39]
Riccardo Mazza and Vania Dimitrova. 2007. CourseVis: A graphical student monitoring tool for supporting instructors in web-based distance courses. International Journal of Human-Computer Studies, Vol. 65, 2 (2007), 125--139. https://doi.org/10.1016/j.ijhcs.2006.08.008
[40]
Philip M McCarthy. 2005. An assessment of the range and usefulness of lexical diversity measures and the potential of the measure of textual, lexical diversity (MTLD). Ph.D. Dissertation. The University of Memphis.
[41]
Philip M McCarthy and Scott Jarvis. 2010. MTLD, vocd-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment. Behavior research methods, Vol. 42, 2 (2010), 381--392.
[42]
Hiroaki Ogata, Chengiu Yin, Moushir M El-Bishouty, and Yoneo Yano. 2010. Computer supported ubiquitous learning environment for vocabulary learning. International Journal of Learning Technology, Vol. 5, 1 (2010), 5--24.
[43]
Selami Ok. 2014. Reflections of ELT Students on Their Progress in Language and Vocabulary Use in Portfolio Process. English Language Teaching, Vol. 7, 2 (2014), 53--62.
[44]
Catherine Plaisant, Brett Milash, Anne Rose, Seth Widoff, and Ben Shneiderman. 1996. LifeLines: visualizing personal histories. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 221--227.
[45]
Yvonne Préfontaine, Judit Kormos, and Daniel Ezra Johnson. 2016. How do utterance measures predict raters' perceptions of fluency in French as a second language? Language Testing, Vol. 33, 1 (2016), 53--73.
[46]
Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics.
[47]
Mohi Reza and Dongwook Yoon. 2021. Designing CAST: a computer-assisted shadowing trainer for self-regulated foreign language listening practice. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1--13.
[48]
Marian J Rossiter. 2009. Perceptions of L2 fluency by native and non-native speakers of English. Canadian Modern Language Review, Vol. 65, 3 (2009), 395--412.
[49]
Peter Skehan et al. 1998. A cognitive approach to language learning .Oxford University Press.
[50]
Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2020. MPNet: Masked and Permuted Pre-training for Language Understanding. In NIPS .
[51]
William L Tarvin and ALI YAHYA AL-ARISHI. 1991. Rethinking Communicative Language-Teaching: Reflection and the EFL Classroom. Tesol Quarterly, Vol. 25, 1 (1991), 9--27.
[52]
Katrien Verbert, Xavier Ochoa, Robin De Croon, Raphael A Dourado, and Tinne De Laet. 2020. Learning analytics dashboards: the past, the present and the future. In Proceedings of the tenth international conference on learning analytics & knowledge. 35--40.
[53]
Stephan Weibelzahl. 2001. Evaluation of adaptive systems. In International Conference on User Modeling. Springer, 292--294.
[54]
DOROTA WERBI'NSKA, ELIF BOZYI?IT, LUÌS GUERRA, Małgorzata Ekiert, and SERHAN KÖSE. 2019. English language teachers' conceptualizations of one-to-one private tutoring: An international phenomenographic study. Glottodidactica. An International Journal of Applied Linguistics, Vol. 46, 2 (2019), 175--196.
[55]
Keith Willey and AP Gardner. 2010a. Improving the standard and consistency of multi-tutor grading in large classes. In ATN Assessment Conference. Institute for Interactive Media and Learning, University of Technology Sydney.
[56]
Keith Willey and Anne Gardner. 2010b. Perceived differences in tutor grading in large classes: Fact or fiction?. In 2010 IEEE Frontiers in Education Conference (FIE). IEEE, S2G--1.
[57]
Keith Willey, Anne Gardner, et al. 2011. Getting tutors on the same page. In Australasian Association for Engineering Education Conference 2011: Developing engineers for social justice: Community involvement, ethics & sustainability 5--7 December 2011, Fremantle, Western Australia. Engineers Australia, 454.
[58]
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020. Transformers: State-of-the-Art Natural Language Processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. Association for Computational Linguistics, 38--45.
[59]
Chris F Wright, Nick Wailes, Greg J Bamber, and Russell D Lansbury. 2017. Beyond national systems, towards a ?gig economy'? A research agenda for international and comparative employment relations. Employee Responsibilities and Rights Journal, Vol. 29, 4 (2017), 247--257.
[60]
Jinyue Xia and David C. Wilson. 2018. Instructor Perspectives on Comparative Heatmap Visualizations of Student Engagement with Lecture Video: Comparative Heatmap Visualizations of Student Video Engagement. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education, SIGCSE 2018, Baltimore, MD, USA, February 21--24, 2018. 251--256. https://doi.org/10.1145/3159450.3159487
[61]
Meng Xia, Yuya Asano, Joseph Jay Williams, Huamin Qu, and Xiaojuan Ma. 2020. Using Information Visualization to Promote Students' Reflection on" Gaming the System" in Online Learning. In Proceedings of the Seventh ACM Conference on Learning@ Scale. 37--49.
[62]
Meng Xia, Mingfei Sun, Huan Wei, Qing Chen, Yong Wang, Lei Shi, Huamin Qu, and Xiaojuan Ma. 2019. Peerlens: Peer-inspired interactive learning path planning in online question pool. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1--12.
[63]
Meng Xia, Yankun Zhao, Mehmet Hamza Erol, Jihyeong Hong, and Juho Kim. 2022. Understanding Distributed Tutorship in Online Language Tutoring. In LAK22: 12th International Learning Analytics and Knowledge Conference. 164--174.
[64]
Hisako Yamashita and Satoko Kato. 2012. The Wheel of Language Learning: A tool to facilitate learner awareness, reflection and action. Advising in language learning: Dialogue, tools and context (2012), 164--169.
[65]
Hui-Chin Yeh and Wei-Yun Lai. 2019. Speaking progress and meaning negotiation processes in synchronous online tutoring. System, Vol. 81 (2019), 179--191.
[66]
Kevin Wai Ho Yung and Ming Ming Chiu. 2020. Secondary school students' enjoyment of English private tutoring: An L2 motivational self perspective. Language Teaching Research (2020), 1362168820962139.
[67]
Kevin Wai Ho Yung and Rui Yuan. 2020. 'The most popular star-tutor of English': Discursive construction of tutor identities in shadow education. Discourse: Studies in the Cultural Politics of Education, Vol. 41, 1 (2020), 153--168.
[68]
Jian Zhao, Christopher Collins, Fanny Chevalier, and Ravin Balakrishnan. 2013. Interactive exploration of implicit and explicit relations in faceted datasets. IEEE Transactions on Visualization and Computer Graphics, Vol. 19, 12 (2013), 2080--2089.
Index Terms
- RLens: A Computer-aided Visualization System for Supporting Reflection on Language Learning under Distributed Tutorship
Recommendations
Understanding Distributed Tutorship in Online Language Tutoring
LAK22: LAK22: 12th International Learning Analytics and Knowledge ConferenceWith the rise of the gig economy, online language tutoring platforms are becoming increasingly popular. They provide temporary and flexible jobs for native speakers as tutors and allow language learners to have one-on-one speaking practices on demand. ...
Mobile blogs in language learning: Making the most of informal and situated learning opportunities
The application of mobile technologies to learning has the potential to facilitate the active participation of learners in the creation and delivery of content. Mobile technologies can also provide a powerful connection between a variety of formal and ...
Supportiveness of language MOOCs for self-regulated learning: a review of commercial language MOOCs on the market
MOOCs have been frequently applied as an effective approach to language education, especially when they can support self-regulated learning. However, few studies have discussed the supportiveness of using MOOCs for language education (i.e. language MOOCs) ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
June 2022
491 pages
ISBN:9781450391580
DOI:10.1145/3491140
- General Chair:
- René F. Kizilcec,
- Program Chairs:
- Katie Davis,
- Xavier Ochoa
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 01 June 2022
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- Institute of Information & commu- nications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT)
Conference
L@S '22
Acceptance Rates
Overall Acceptance Rate 117 of 440 submissions, 27%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 236Total Downloads
- Downloads (Last 12 months)61
- Downloads (Last 6 weeks)6
Reflects downloads up to 25 Nov 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in