short-paper

Effects of real-time transcription on non-native speaker's comprehension in computer-mediated communications

Authors:

Yingxin Pan,

Danning Jiang,

Michael Picheny,

Yong QinAuthors Info & Claims

CHI '09: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Pages 2353 - 2356

https://doi.org/10.1145/1518701.1519061

Published: 04 April 2009 Publication History

Get Access

Abstract

We performed an empirical study to understand the relative contributions of real-time transcription to a non-native speaker's comprehension in audio/video meetings. 48 participants were assigned to 2 presentation modes (audio, audio+video) and 3 transcription modes (no transcript, real-time transcripts in the streaming mode, transcripts with all past records) in a 3x2 factorial experimental design. The results suggest that comprehension can be significantly improved for both audio and audio+video conditions when real-time transcription is provided. Also, the participants reported positive subjective responses to the presence of real-time transcription in terms of usefulness, preference, and willingness to use such a feature if provided. No cognitive load issues were reported by the participants in the ability to synthesize across modalities. Implications for system development and design, as well as future work utilizing automation speech recognition to provide the transcripts are discussed.

References

[1]

Tyler, M.D. The Effect of Background Knowledge on First and Second Language Comprehension Difficulty. In Proc. of ICSLP 1998 (International Conference on Spoken Language Processing.)

Google Scholar

[2]

Nakamura S., Markov K., Nakaiwa H., et al. The ATR Multilingual Speech-to-Speech Translation System. IEEE Transactions on Audio, Speech, and Language Processing 10, 2 (2006), 365--376.

Digital Library

Google Scholar

[3]

Imoto K., Sasajima M., Shimomori T., et al. A Multi-modal Supporting Tool for Multi-lingual Communication by Inducing Partner's Reply. In Proc. IUI'2006, ACM Press (2006), 330--332.

Digital Library

Google Scholar

[4]

Chen S., Kingsbury B., Mangu L., et al. Advances in Speech Transcription at IBM under the DAPAR EARS Program. IEEE Transactions on Audio, Speech, and Language Processing 14, 5 (2006), 1596--1608.

Digital Library

Google Scholar

[5]

Cui X., Gu L., Xiang B., et al. Developing High Performance ASR in the IBM Multilingual Speech-to-Speech Translation System. In Proc. ICASSP 2008 (International Conference on Acoustics, Speech, and Signal Processing), IEEE Press (2008), 5121--5124.

Google Scholar

[6]

Markham P.L., Peter L. A., McCarthy T.J. The Effects of Native Language vs. Target Language Captions on Foreign Language Students' DVD Video Comprehension. Foreign Language Annals 34, 5 (2001), 439--445.

Crossref

Google Scholar

[7]

Jin Y., Psychological Measurement. East China Normal University Press, China, 2005.

Google Scholar

[8]

Veinott, E.S., Fu, X.L., Olsen J, et al. Video Helps Remote Work: Speaker Who Need to Negotiate Common Ground Benefit from Seeing Each Other. In Proc. CHI 1999, ACM Press (1999), 302--309.

Digital Library

Google Scholar

Cited By

View all

Kim SPark YAhn DKwak JKim J(2024)Is the Same Performance Really the Same?: Understanding How Listeners Perceive ASR Results Differently According to the Speaker's AccentProceedings of the ACM on Human-Computer Interaction10.1145/36410088:CSCW1(1-22)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3641008
Hautasaari AAramaki MChujo RNaemura T(2024)EmoScribe Camera: A Virtual Camera System to Enliven Online Conferencing with Automatically Generated Emotional Text CaptionsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650987(1-7)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650987
Seitz JBenke IHeinzl AMaedche A(2024)The Impact of Video Meeting Systems on Psychological User StatesInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2023.103178182:COnline publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.ijhcs.2023.103178
Show More Cited By

Index Terms

Effects of real-time transcription on non-native speaker's comprehension in computer-mediated communications
1. Human-centered computing
  1. Collaborative and social computing
    1. Collaborative and social computing theory, concepts and paradigms
      1. Computer supported cooperative work
2. Social and professional topics
  1. Professional topics
    1. Computing and business
      1. Computer supported cooperative work

Recommendations

Effects of automated transcription quality on non-native speakers' comprehension in real-time computer-mediated communication
CHI '10: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Real-time transcription has been shown to be valuable in facilitating non-native speakers' comprehension in real-time communication. Automated speech recognition (ASR) technology is a critical ingredient for its practical deployment. This paper presents ...
The Lombard intelligibility benefit of native and non-native speech for native and non-native listeners
Highlights
- We compared native English and non-native (Dutch) Lombard and plain speech.
- ...
Abstract
Speech produced in noise (Lombard speech) is more intelligible than speech produced in quiet (plain speech). Previous research on the Lombard intelligibility benefit focused almost entirely on how native speakers produce and perceive ...
Effects of automated transcription delay on non-native speakers' comprehension in real-time computer-mediated communication
INTERACT'11: Proceedings of the 13th IFIP TC 13 international conference on Human-computer interaction - Volume Part I

Real-time transcription generated by automated speech recognition (ASR) technologies with a reasonably high accuracy has been demonstrated to be valuable in facilitating non-native speakers' comprehension in real-time communication. Besides errors, time ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

CHI '09: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

April 2009

2426 pages

ISBN:9781605582467

DOI:10.1145/1518701

General Chairs:
Dan R. Olsen
Brigham Young University
,
Richard B. Arthur
Brigham Young University
,
Program Chairs:
Ken Hinckley
Microsoft Research
,
Meredith Ringel Morris
Microsoft Research
,
Scott Hudson
Carnegie Mellon University
,
Saul Greenberg
University of Calgary

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 April 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

CHI '09

Sponsor:

CHI '09: CHI Conference on Human Factors in Computing Systems

April 4 - 9, 2009

MA, Boston, USA

Acceptance Rates

CHI '09 Paper Acceptance Rate 277 of 1,130 submissions, 25%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

19
Total Citations
View Citations
486
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)0

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Kim SPark YAhn DKwak JKim J(2024)Is the Same Performance Really the Same?: Understanding How Listeners Perceive ASR Results Differently According to the Speaker's AccentProceedings of the ACM on Human-Computer Interaction10.1145/36410088:CSCW1(1-22)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3641008
Hautasaari AAramaki MChujo RNaemura T(2024)EmoScribe Camera: A Virtual Camera System to Enliven Online Conferencing with Automatically Generated Emotional Text CaptionsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650987(1-7)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650987
Seitz JBenke IHeinzl AMaedche A(2024)The Impact of Video Meeting Systems on Psychological User StatesInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2023.103178182:COnline publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.ijhcs.2023.103178
Duan WYamashita NFussell S(2019)Increasing Native Speakers' Awareness of the Need to Slow Down in Multilingual Conversations Using a Real-Time Speech SpeedometerProceedings of the ACM on Human-Computer Interaction10.1145/33592733:CSCW(1-25)Online publication date: 7-Nov-2019
https://dl.acm.org/doi/10.1145/3359273
Chen MYamashita NWang H(2018)Beyond Lingua FrancaProceedings of the ACM on Human-Computer Interaction10.1145/32743032:CSCW(1-22)Online publication date: 1-Nov-2018
https://dl.acm.org/doi/10.1145/3274303
Gao GFussell SMark GFussell SLampe Cschraefel mHourcade JAppert CWigdor D(2017)A Kaleidoscope of LanguagesProceedings of the 2017 CHI Conference on Human Factors in Computing Systems10.1145/3025453.3025839(760-772)Online publication date: 2-May-2017
https://dl.acm.org/doi/10.1145/3025453.3025839
Pan MYamashita NWang HLee CPoltrock SBarkhuus LBorges MKellogg W(2017)Task RebalancingProceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing10.1145/2998181.2998304(310-321)Online publication date: 25-Feb-2017
https://dl.acm.org/doi/10.1145/2998181.2998304
Arawjo IYoon DGuimbretière FLee CPoltrock SBarkhuus LBorges MKellogg W(2017)TypeTalkerProceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing10.1145/2998181.2998260(1970-1981)Online publication date: 25-Feb-2017
https://dl.acm.org/doi/10.1145/2998181.2998260
He HYamashita NHautasaari ACao XHuang ELee CPoltrock SBarkhuus LBorges MKellogg W(2017)Why Did They Do That?Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing10.1145/2998181.2998205(297-309)Online publication date: 25-Feb-2017
https://dl.acm.org/doi/10.1145/2998181.2998205
Jamieson JYamashita NBoase J(2017)Identifying Support Opportunities for Foreign Students: Disentangling Language and Non-language Problems Among a Unique PopulationHuman-Computer Interaction - INTERACT 201710.1007/978-3-319-67684-5_3(33-53)Online publication date: 20-Sep-2017
https://doi.org/10.1007/978-3-319-67684-5_3
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Effects of automated transcription quality on non-native speakers' comprehension in real-time computer-mediated communication

The Lombard intelligibility benefit of native and non-native speech for native and non-native listeners

Effects of automated transcription delay on non-native speakers' comprehension in real-time computer-mediated communication

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Upcoming Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations