research-article

The Effect of a Robot's Gestures and Adaptive Tutoring on Children's Acquisition of Second Language Vocabularies

Authors:

Thorsten Schodde,

Bram Willemsen,

Kirsten Bergmann,

Mirjam de Haas,

Paul VogtAuthors Info & Claims

HRI '18: Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction

Pages 50 - 58

https://doi.org/10.1145/3171221.3171277

Published: 26 February 2018 Publication History

Abstract

This paper presents a study in which children, four to six years old, were taught words in a second language by a robot tutor. The goal is to evaluate two ways for a robot to provide scaffolding for students: the use of iconic gestures, combined with adaptively choosing the next learning task based on the child»s past performance. The results show a positive effect on long-term memorization of novel words, and an overall higher level of engagement during the learning activities when gestures are used. The adaptive tutoring strategy reduces the extent to which the level of engagement is diminishing during the later part of the interaction.

References

[1]

Martha W. Alibali and Mitchell J. Nathan. 2007. Teachers' Gestures as a Means of Scaffolding Students' Understanding: Evidence From an Early Algebra Lesson. Video Research in the Learning Sciences 39, 5 (2007), 349--366.

[2]

Kirsten Bergmann and Manuela Macedonia. 2013. A virtual agent as vocabulary trainer: iconic gestures help to improve learners memory performance. In International Workshop on Intelligent Virtual Agents. Springer, 139--148.

[3]

Phyllis C. Blumenfeld, Toni M. Kempler, and Joseph S. Krajcik. 2005. Motivation and Cognitive Engagement in Learning Environments. Cambridge University Press, Cambridge, Chapter 28, 475--488.

[4]

Paul Bremner and Ute Leonards. 2016. Iconic gestures for robot avatars, recognition and integration with speech. Frontiers in Psychology 7 (feb 2016), 183.

[5]

Susan Wagner Cook, Zachary Mitchell, and Susan Goldin-Meadow. 2008. Gesturing makes learning last. Cognition 106, 2 (2008), 1047--1058. arXiv:NIHMS150003

[6]

Albert T. Corbett and John R. Anderson. 1994. Knowledge tracing: Modeling the acquisition of procedural knowledge. User modeling and user-adapted interaction 4, 4 (1994), 253--278.

[7]

Scotty Craig, Arthur Graesser, Jeremiah Sullins, and Barry Gholson. 2004. Affect and learning: an exploratory look into the role of affect in learning with AutoTutor. Journal of educational media 29, 3 (2004), 241--250.

[8]

Jacqueline A. de Nooijer, Tamara van Gog, Fred Paas, and Rolf A. Zwaan. 2013. Effects of imitating gestures during encoding or during retrieval of novel verbs on children's test performance. Acta Psychologica 144, 1 (2013), 173--179.

[9]

Jan de Wit, Thorsten Schodde, Bram Willemsen, Kirsten Bergmann, Mirjam de Haas, Stefan Kopp, Emiel Krahmer, and Paul Vogt. 2017. Exploring the Effect of Gestures and Adaptive Tutoring on Children's Comprehension of L2 Vocabularies. In Proceedings of the Workshop R4L at ACM/IEEE HRI 2017.

[10]

Katinka Dijkstra and Lysanne Post. 2015. Mechanisms of embodiment. 6, OCT (2015), 1525.

[11]

Goren Gordon and Cynthia Breazeal. 2015. Bayesian Active Learning-based Robot Tutor for Children's Word-reading Skills. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI'15). AAAI Press, 1343--1349.

Digital Library

[12]

Lea A. Hald, Jacqueline de Nooijer, Tamara van Gog, and Harold Bekkering. 2016. Optimizing Word Learning via Links to Perceptual and Motoric Experience. Educational Psychology Review 28, 3 (2016), 495--522.

[13]

Autumn B. Hostetter. 2011. When do gestures communicate? A meta-analysis. Psychological Bulletin 137, 2 (2011), 297--315.

[14]

Tanja Käser, Severin Klingler, Alexander Gerhard Schwing, and Markus Gross. 2014. Beyond knowledge tracing: Modeling skill topologies with bayesian networks. In International Conference on Intelligent Tutoring Systems. Springer, 188-- 198.

Digital Library

[15]

Spencer D. Kelly, Tara McDevitt, and Megan Esch. 2009. Brief training with co-speech gesture lends a hand to word learning in a foreign language. Language and Cognitive Processes 24, 2 (2009), 313--334.

[16]

James Kennedy, Paul Baxter, and Tony Belpaeme. 2015. The Robot Who Tried Too Hard: Social Behaviour of a Robot Tutor Can Negatively Affect Child Learning. In Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction. 67--74.

Digital Library

[17]

James Kennedy, Severin Lemaignan, Caroline Montassier, Pauline Lavalade, Bahar Irfan, Fotios Papadopoulos, Emmanuel Senft, and Tony Belpaeme. 2017. Child Speech Recognition in Human-Robot Interaction : Evaluations and Recommendations. Proc. of the ACM/IEEE International Conference on Human-Robot Interaction (HRI) (2017), 82--90.

Digital Library

[18]

S. Leitner. 1972. So lernt man Lernen: Der Weg zum Erfolg {Learning to learn: The road to success}. Freiburg: Herder.

[19]

Daniel Leyzberg, Samuel Spaulding, and Brian Scassellati. 2014. Personalizing robot tutors to individuals' learning differences. In Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction. ACM, 423--430.

Digital Library

[20]

Daniel Leyzberg, Samuel Spaulding, Mariya Toneva, and Brian Scassellati. 2012. The Physical Presence of a Robot Tutor Increases Cognitive Learning Gains. 34th Annual Conference of the Cognitive Science Society 34, 1 (jan 2012), 1882--1887. https://doi.org/ISBN978-0--9768318--8--4

[21]

Manuela Macedonia, Karsten Müller, and Angela D. Friederici. 2011. The impact of iconic gestures on foreign language word learning and its neural substrate. Human Brain Mapping 32, 6 (2011), 982--998.

[22]

Panos Markopoulos, Janet C. Read, Stuart MacFarlane, and Johanna Hoysniemi. 2008. Evaluating Children's Interactive Products: Principles and Practices for Interaction Designers. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, Chapter 1, 3--18.

Digital Library

[23]

David McNeill. 1985. So you think gestures are nonverbal? Psychological Review 92, 3 (1985), 350--371.

[24]

Omar Mubin, Catherine J. Stevens, Suleman Shahid, Abdullah Al Mahmud, and Jian-Jie Dong. 2013. A Review of the Applicability of Robots in Education. Technology for Education and Learing 1 (2013), 209--0015.

[25]

Meredith L. Rowe, Rebecca D. Silverman, and Bridget E. Mullan. 2013. The role of pictures and gestures as nonverbal aids in preschoolers' word learning in a novel language. Contemporary Educational Psychology 38, 2 (2013), 109--117.

[26]

Thorsten Schodde, Kirsten Bergmann, and Stefan Kopp. 2017. Adaptive Robot Language Tutoring Based on Bayesian Knowledge Tracing and Predictive Decision-Making. In Proceedings of ACM/IEEE HRI 2017. ACM Press, 128--136.

Digital Library

[27]

Samuel Spaulding, Goren Gordon, and Cynthia Breazeal. 2016. Affect-Aware Student Models for Robot Tutors. In Proceedings of the 2016 International Conference on Autonomous Agents&Multiagent Systems (AAMAS '16). International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, 864--872.

Digital Library

[28]

Marion Tellier. 2008. The effect of gestures on second language memorisation by young children. Gesture 8, 2 (2008), 219--235.

[29]

Janneke van de Pol, Monique Volman, and Jos Beishuizen. 2010. Scaffolding in teacher-student interaction: A decade of research. (2010), 271--296 pages. arXiv:arXiv:1002.2562v1

[30]

Elisabeth T. van Dijk, Elena Torta, and Raymond H. Cuijpers. 2013. Effects of Eye Contact and Iconic Gestures on Message Retention in Human-Robot Interaction. International Journal of Social Robotics 5, 4 (2013), 491--501.

[31]

Paul Vogt, Mirjam De Haas, Chiara De Jong, Peta Baxter, and Emiel Krahmer. 2017. Child-Robot Interactions for Second Language Tutoring to Preschool Children. Frontiers in human neuroscience 11, March (2017), 1--7.

[32]

Lev Vygotsky. 1978. Mind in society: The development of higher psychological processes. Harvard University Press, Cambridge, MA

Cited By

Derakhshan ATeo TSaeedy Robat EJanebi Enayat MJahanbakhsh A(2024)Robot-Assisted Language Learning: A Meta-AnalysisReview of Educational Research10.3102/00346543241247227Online publication date: 1-May-2024
https://doi.org/10.3102/00346543241247227
Wang FCheung A(2024)Robots’ Social Behaviors for Language Learning: A Systematic Review and Meta-AnalysisReview of Educational Research10.3102/00346543231216437Online publication date: 3-Jan-2024
https://doi.org/10.3102/00346543231216437
Wu XLi R(2024)Effects of Robot-Assisted Language Learning on English-as-a-Foreign-Language Skill DevelopmentJournal of Educational Computing Research10.1177/0735633123122617162:4(1010-1034)Online publication date: 10-Jan-2024
https://doi.org/10.1177/07356331231226171
Show More Cited By

Index Terms

The Effect of a Robot's Gestures and Adaptive Tutoring on Children's Acquisition of Second Language Vocabularies

Recommendations

Adaptive Robot Language Tutoring Based on Bayesian Knowledge Tracing and Predictive Decision-Making
HRI '17: Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction

In this paper, we present an approach to adaptive language tutoring in child-robot interaction. The approach is based on a dynamic probabilistic model that represents the inter-relations between the learner's skills, her observed behaviour in tutoring ...
Adaptive Robot Second Language Tutoring for Children
HRI '18: Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction

Empirical evidence has demonstrated that learning with and from a physically present, interactive robot can be more effective than learning from classical on-screen media [6, 8]. In the L2TOR project wework on using the robot Nao to support second ...
A Robot Teaching Young Children a Second Language: The Effect of Multiple Interactions on Engagement and Performance
HRI '18: Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction

This paper explores the use of a social robot for one-on-one tutoring, in a study in which 15 children participated in four second-language tutoring sessions. Specifically, changes across sessions are measured on two dimensions: engagement and ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

HRI '18: Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction

February 2018

468 pages

ISBN:9781450349536

DOI:10.1145/3171221

General Chairs:
Takayuki Kanda
ATR, Japan
,
Selma Ŝabanović
Indiana University Bloomington, USA
,
Program Chairs:
Guy Hoffman
Cornell University, USA
,
Adriana Tapus
ENSTA-ParisTech, France

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGAI: ACM Special Interest Group on Artificial Intelligence
SIGCHI: ACM Special Interest Group on Computer-Human Interaction
IEEE-RAS: Robotics and Automation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 February 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Horizon2020 L2TOR project

Conference

HRI '18

Sponsor:

SIGAI
SIGCHI
IEEE-RAS

HRI '18: ACM/IEEE International Conference on Human-Robot Interaction

March 5 - 8, 2018

IL, Chicago, USA

Acceptance Rates

HRI '18 Paper Acceptance Rate 49 of 206 submissions, 24%;

Overall Acceptance Rate 268 of 1,124 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

66
Total Citations
View Citations
1,211
Total Downloads

Downloads (Last 12 months)96
Downloads (Last 6 weeks)9

Reflects downloads up to 16 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Derakhshan ATeo TSaeedy Robat EJanebi Enayat MJahanbakhsh A(2024)Robot-Assisted Language Learning: A Meta-AnalysisReview of Educational Research10.3102/00346543241247227Online publication date: 1-May-2024
https://doi.org/10.3102/00346543241247227
Wang FCheung A(2024)Robots’ Social Behaviors for Language Learning: A Systematic Review and Meta-AnalysisReview of Educational Research10.3102/00346543231216437Online publication date: 3-Jan-2024
https://doi.org/10.3102/00346543231216437
Wu XLi R(2024)Effects of Robot-Assisted Language Learning on English-as-a-Foreign-Language Skill DevelopmentJournal of Educational Computing Research10.1177/0735633123122617162:4(1010-1034)Online publication date: 10-Jan-2024
https://doi.org/10.1177/07356331231226171
Wolfert PDe Gersem LJanssens RBelpaeme TGrollman DBroadbent EJu WSoh HWilliams T(2024)Multi-modal Language Learning: Explorations on learning Japanese VocabularyCompanion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610978.3640685(1129-1133)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610978.3640685
Zhu YWen RWilliams TGrollman DBroadbent EJu WSoh HWilliams T(2024)Robots for Social Justice (R4SJ): Toward a More Equitable Practice of Human-Robot InteractionProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3634944(850-859)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610977.3634944
Gvirsman OGordon GGrollman DBroadbent EJu WSoh HWilliams T(2024)Effect of Social Robot's Role and Behavior on Parent-Toddler InteractionProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3634928(222-230)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610977.3634928
Janssen SSchadenberg B(2024)A Psychological Need-Fulfillment Perspective for Designing Social Robots that Support Well-BeingInternational Journal of Social Robotics10.1007/s12369-024-01102-816:5(857-878)Online publication date: 25-Feb-2024
https://doi.org/10.1007/s12369-024-01102-8
Fung KLee LSin KSong SQu H(2024)Humanoid robot-empowered language learning based on self-determination theoryEducation and Information Technologies10.1007/s10639-024-12570-w29:14(18927-18957)Online publication date: 15-Mar-2024
https://doi.org/10.1007/s10639-024-12570-w
Yun HHübert HPinkwart NHafner V(2024)Design Based Research of Multimodal Robotic Learning CompanionsArtificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky10.1007/978-3-031-64312-5_12(97-104)Online publication date: 2-Jul-2024
https://doi.org/10.1007/978-3-031-64312-5_12
Geçkin V(2023)Assessing the Learning Outcomes of Robot-Assisted Second Language LearningHandbook of Research on Perspectives in Foreign Language Assessment10.4018/978-1-6684-5660-6.ch005(71-88)Online publication date: 13-Jan-2023
https://doi.org/10.4018/978-1-6684-5660-6.ch005
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents