research-article

Open access

Unmet Needs and Opportunities for Mobile Translation AI

Authors:

Daniel J. Liebling,

Aaron Donsbach,

Lindsey BoranAuthors Info & Claims

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Pages 1 - 13

https://doi.org/10.1145/3313831.3376261

Published: 23 April 2020 Publication History

All formats PDF

Abstract

Translation apps and devices are often presented in the context of providing assistance while traveling abroad. However, the spectrum of needs for cross-language communication is much wider. To investigate these needs, we conducted three studies with populations spanning socioeconomic status and geographic regions: (1) United States-based travelers, (2) migrant workers in India, and (3) immigrant populations in the United States. We compare frequent travelers' perception and actual translation needs with those of the two migrant communities. The latter two, with low language proficiency, have the greatest translation needs to navigate their daily lives. However, current mobile translation apps do not meet these needs. Our findings provide new insights on the usage practices and limitations of mobile translation tools. Finally, we propose design implications to help apps better serve these unmet needs.

References

[1]

Denice Adkins and Heather Moulaison Sandy. 2019. Information behavior and ICT use of Latina immigrants to the U.S. Midwest. Information Processing & Management (2019), 102072.

Digital Library

[2]

E. Annamalai. 2003. Medium of Power: The Question of English in India. Routledge, 177--194.

[3]

Luis Fernando Baron and Ricardo Gomez. 2017. Living in the Limits: Migration and Information Practices of Undocumented Latino Migrants. In Information and Communication Technologies for Development (ICT4D), Jyoti Choudrie, M. Sirajul Islam, Fathul Wahid, Julian M. Bass, and Johanes Eka Priyatma (Eds.), Vol. 504. Springer International, Cham, Switzerland, 147--158.

[4]

John W Berry. 1997. Immigration, acculturation, and adaptation. Applied psychology 46, 1 (1997), 5--34.

[5]

Dan Bohus, Sean Andrist, and Mihai Jalobeanu. 2017. Rapid Development of Multimodal Interactive Systems: A Demonstration of Platform for Situated Intelligence. In Proceedings of the 19th ACM International Conference on Multimodal Interaction (ICMI '17). ACM, NY, NY, USA, 493--494.

Digital Library

[6]

United States Census Bureau. 2017. Detailed household language by household limited English speaking status. American Community Survey 1-year Estimates (2017).

[7]

Jack Burston. 2014. MALL: the pedagogical challenges. Computer Assisted Language Learning 27, 4 (2014), 344--357.

[8]

Barry R Chiswick. 1978. A longitudinal analysis of the occupational mobility of immigrants. In Proceedings of the 30th Annual Industrial Relations Research Association Meeting, Barbara Dennis (Ed.). University of Wisconsin Press, 20--27.

[9]

Leigh Clark, Philip Doyle, Diego Garaialde, Emer Gilmartin, Stephan Schl ogl, Jens Edlund, Matthew Aylett, João Cabral, Cosmin Munteanu, Justin Edwards, and Benjamin R Cowan. 2019. The State of Speech in HCI: Trends, Themes and Challenges. Interacting with Computers (09 2019).

[10]

Carrie Demmans Epp. 2016. Noticing: ELL use of MALL for filling the gap. In Proceedings of 2016 Computer-Assisted Language Instruction Consortium (CALICO).

[11]

Carrie Demmans Epp. 2017. Migrants and Mobile Technology Use: Gaps in the Support Provided by Current Tools. Journal of interactive media in education 2017, 1 (2017).

[12]

David M. Eberhard, Gary F. Simons, and Charles D. Fennig (Eds.). Ethnologue: Languages of the World (22 ed.). SIL International. http://www.ethnologue.com

[13]

Carrie Demmans Epp. 2016. English Language Learner Experiences of Formal and Informal Learning Environments. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (LAK '16). ACM, NY, NY, USA, 231--235.

Digital Library

[14]

Victor Fragoso, Steffen Gauglitz, Jim Kleban, Shane Zamora, and Matthew Turk. 2012. TranslatAR: A Mobile Augmented Reality Translator on the Nokia N900. In 2011 IEEE Workshop on Applications of Computer Vision (WACV). 497--502.

Digital Library

[15]

Vittorio Fuccella, Poika Isokoski, and Benoit Martin. 2013. Gestures and Widgets: Performance in Text Editing on Multi-touch Capable Mobile Devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, NY, NY, USA, 2785--2794.

Digital Library

[16]

Masaaki Fukumoto. 2018. SilentVoice: Unnoticeable Voice Input by Ingressive Speech. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology (UIST '18). ACM, NY, NY, USA, 237--246.

Digital Library

[17]

Ge Gao, Naomi Yamashita, Ari MJ Hautasaari, Andy Echenique, and Susan R. Fussell. 2014. Effects of Public vs. Private Automated Transcripts on Multiparty Communication Between Native and Non-native English Speakers. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, NY, NY, USA, 843--852.

Digital Library

[18]

Roberto G. Gonzales and Steven Raphael. 2017. Illegality: A Contemporary Portrait of Immigration. RSF: The Russell Sage Foundation Journal of the Social Sciences 3, 4 (2017), 1--17. http://www.jstor.org/stable/10.7758/rsf.2017.3.4.01

[19]

Tamy Guberek, Allison McDonald, Sylvia Simioni, Abraham H. Mhaidli, Kentaro Toyama, and Florian Schaub. 2018. Keeping a Low Profile?: Technology, Risk and Privacy Among Undocumented Immigrants. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA.

Digital Library

[20]

Kotaro Hara and Shamsi T. Iqbal. 2015. Effect of Machine Translation in Interlingual Conversation: Lessons from a Formative Study. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, NY, NY, USA, 3473--3482.

Digital Library

[21]

W John Hutchins. 1995. Machine translation: A brief history. In Concise history of the language sciences: from the Sumerians to the cognitivists, E. F. K. Koerner and R. E. Asher (Eds.). Pergamon Press, 431--445. http://hutchinsweb.me.uk/ConcHistoryLangSci-1995.pdf

[22]

J Jeyaranjan. 2017. The Life and Times of Migrant Workers in Chennai. In Rural Labour Mobility in Times of Structural Transformation, D. Reddy and K. Sarap (Eds.). Palgrave Macmillan, 299--326.

[23]

Hyun soo Ji and Joong seek Lee. 2016. Exploring User Behavior on Mobile Copy-Paste. In Proceedings of HCI Korea (HCIK '16). Hanbit Media, Inc., South Korea, 327--332.

Digital Library

[24]

Amy K. Karlson, A.J. Bernheim Brush, and Stuart Schechter. 2009. Can I Borrow Your Phone?: Understanding Concerns when Sharing Mobile Phones. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). ACM, NY, NY, USA, 1647--1650.

Digital Library

[25]

Naveena Karusala, Aditya Vishwanath, Aditya Vashistha, Sunita Kumar, and Neha Kumar. 2018. ?Only if You Use English You Will Get to More Things": Using Smartphones to Navigate Multilingualism. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA.

Digital Library

[26]

Agnes Kukulska-Hulme. 2013. Re-skilling language learners for a mobile world. (2013). http://www.tirfonline.org/english-in-the-workforce/ mobileassisted-language-learning/ re-skilling-language-learners-for-a-mobile-world/

[27]

Agnes Kukulska-Hulme and Lesley Shield. 2008. An overview of mobile assisted language learning: From content delivery to supported collaboration and interaction. ReCALL 20 (2008), 271--289.

[28]

Jessa Lingel. 2011. Information tactics of immigrants in urban environments. Information Research 16, 4 (2011). http://InformationR.net/ir/16--4/paper500.html

[29]

William D. Lopez, Daniel J. Kruger, Jorge Delva, Mikel Llanes, Charo Ledón, Adreanne Waller, Melanie Harner, Ramiro Martinez, Laura Sanders, Margaret Harner, and Barbara Israel. 2017. Health Implications of an Immigration Raid: Findings from a Latino Community in the Midwestern United States. Journal of Immigrant and Minority Health 19, 3 (01 June 2017), 702--708.

[30]

Alejandro Morales and William E. Hanson. 2005. Language Brokering: An Integrative Review of the Literature. Hispanic Journal of Behavioral Sciences 27, 4 (2005), 471--503.

[31]

D.A. Norman and S.W. Draper. 1986. User Centered System Design: New Perspectives on Human-computer Interaction. Taylor & Francis. https://books.google.ch/books?id=Qz5jQgAACAAJ

[32]

Agnieszka Palalas. 2011. Mobile-assisted language learning: Designing for your students. Second language teaching and learning with technology: views of emergent researchers (2011), 71--94.

[33]

Agnieszka Palalas. 2015. The ecological perspective on the 'anytime anyplace' of Mobile-Assisted Language Learning. Technologie mobilne w ksztaceniu jezykowym (2015), 29--48.

[34]

Mei-Hua Pan, Naomi Yamashita, and Hao-Chuan Wang. 2017. Task Rebalancing: Improving Multilingual Communication with Native Speakers-Generated Highlights on Automated Transcripts. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '17). ACM, NY, NY, USA, 310--321.

Digital Library

[35]

Christin Mathew Philip. 2016. Tamil Nadu now home to 1 million migrant workers: Study. Times of India (February 6 2016). https://timesofindia.indiatimes.com/city/chennai/ Tamil-Nadu-now-home-to-1-million-migrant-workers-Study/ articleshow/50861647.cms

[36]

Rohit Prasad, Prem Natarajan, David Stallard, Shirin Saleem, Shankar Ananthakrishnan, Stavros Tsakalidis, Chia lin Kao, Fred Choi, Ralf Meermeier, Mark Rawls, Jacob Devlin, Kriste Krstovski, and Aaron Challenner. 2013. BBN TransTalk: Robust multilingual two-way speech-to-speech translation for mobile platforms. Computer Speech & Language 27, 2 (2013), 475--491.

Digital Library

[37]

Nithya Sambasivan, Amna Batool, Nova Ahmed, Tara Matthews, Kurt Thomas, Laura Sanely Gaytán-Lugo, David Nemer, Elie Bursztein, Elizabeth Churchill, and Sunny Consolvo. 2019. "They Don't Leave Us Alone Anywhere We Go";: Gender and Digital Abuse in South Asia. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, NY, NY, USA.

Digital Library

[38]

Karen Schwartz. 2019. Move Over, Google Translate: Here Come A.I. Earbuds. The New York Times (10 November 2019), TR--2. https://nyti.ms/2pSONID

[39]

Julie Anne Séguin, Alec Scharff, and Kyle Pedersen. 2019. Triptech: A Method for Evaluating Early Design Concepts. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19). ACM, NY, NY, USA.

Digital Library

[40]

Frank Seide, Gang Li, and Dong Yu. 2011. Conversational Speech Transcription Using Context-Dependent Deep Neural Networks. In Proceedings of Interspeech 2011. 437--440.

[41]

Roger C. Shank. 1982. Dynamic memory: A theory of reminding and learning in computers and people. Cambridge University Press.

[42]

JongHo Shin, Panayiotis G. Georgiou, and Shrikanth Narayanan. 2013. Enabling effective design of multimodal interfaces for speech-to-speech translation system: An empirical study of longitudinal user behaviors over time and user strategies for coping with errors. Computer Speech & Language 27, 2 (2013), 554--571.

Digital Library

[43]

Sangmi Shin, Eric T. Matson, Jinok Park, Bowon Yang, Juhee Lee, and Jin-Woo Jung. 2015. Speech-to-speech translation humanoid robot in doctor's office. (2015), 484--489.

[44]

Minhyang (Mia) Suh and Gary Hsieh. 2019. The "Had Mores": Exploring korean immigrants' information behavior and ICT usage when settling in the United States. Journal of the Association for Information Science and Technology 70, 1 (2019), 38--48.

Digital Library

[45]

Paul Taylor, Mark Hugo Lopez, Jeffrey S Passel, and Seth Motel. 2011. Unauthorized immigrants: Length of residency, patterns of parenthood. (2011). http://assets.pewresearch.org/wp-content/uploads/ sites/7/2011/12/Unauthorized-Characteristics.pdf

[46]

John Traxler. 2013. Mobile learning for languages: Can the past speak to the future? (2013). http://www.tirfonline.org/english-in-theworkforce/ mobile-assisted-language-learning/

[47]

United States Census Bureau. 2017. Geographic mobility by selected characteristics in The United States. American Community Survey 1-year Estimates (2017).

[48]

United States Department of State. 2019. Passport Statistics. (2019). https://travel.state.gov/content/ travel/en/passports/after/passport-statistics.html

[49]

U.S. Department of Commerce, International Trade Administration, National Travel and Tourism Office. 2019. U.S. Citizen Traffic to Overseas Regions, Canada & Mexico 2018. (February 2019). https://travel.trade.gov/view/m-2018-O-001/index.html

[50]

Ron J. Weiss, Jan Chorowski, Navdeep Jaitly, Yonghui Wu, and Zhifeng Chen. 2017. Sequence-to-Sequence Models Can Directly Translate Foreign Speech. In Proceedings of Interspeech 2017. 2625--2629.

[51]

Chris Wendt and Will Lewis. 2016. Conversational Speech Translation - Challenges and Techniques. TAUS Forum Tokyo. (April 4 2016).

[52]

Shuang-Hong Yang, Ryen W. White, and Eric Horvitz. 2013. Pursuing Insights About Healthcare Utilization via Geocoded Search Queries. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '13). ACM, NY, NY, USA, 993--996.

Digital Library

[53]

Xuesong Yang, Karthik Audhkhasi, Andrew Rosenberg, Samuel Thomas, Bhuvana Ramabhadran, and Mark Hasegawa-Johnson. 2018. Joint Modeling of Accents and Acoustics for Multi-Accent Speech Recognition. In Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

Digital Library

Cited By

Adjagbodjou AKaufman G(2024)Envisioning Support-Centered Technologies for Language Practice and Use: Needs and Design Opportunities for Immigrant English Language Learners (ELLs)Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642236(1-15)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642236
Calambur VJun DSchiaffino MZhang ZHuh-Yoo J(2024)A case for "little English" in Nurse Notes from the Telehealth Intervention Program for Seniors: Implications for Future Design and ResearchProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641961(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641961
Lee NLee K(2024)Travelers’ viewpoints on machine translation using Q methodology: a perspective of consumption value theoryInformation Technology & Tourism10.1007/s40558-024-00296-xOnline publication date: 13-Aug-2024
https://doi.org/10.1007/s40558-024-00296-x
Show More Cited By

Index Terms

Unmet Needs and Opportunities for Mobile Translation AI
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
      1. Machine translation
2. Human-centered computing
  1. Collaborative and social computing
    1. Collaborative and social computing design and evaluation methods
      1. Ethnographic studies
  2. Human computer interaction (HCI)
    1. HCI design and evaluation methods
      1. Field studies

Recommendations

Language Modeling for Syntax-Based Machine Translation Using Tree Substitution Grammars: A Case Study on Chinese-English Translation

The poor grammatical output of Machine Translation (MT) systems appeals syntax-based approaches within language modeling. However, previous studies showed that syntax-based language modeling using (Context-Free) Treebank Grammars was not very helpful in ...
Large aligned treebanks for syntax-based machine translation

We present a collection of parallel treebanks that have been automatically aligned on both the terminal and the non-terminal constituent level for use in syntax-based machine translation. We describe how they were constructed and applied to a syntax- ...
Post-Ordering by Parsing with ITG for Japanese-English Statistical Machine Translation

Word reordering is a difficult task for translation between languages with widely different word orders, such as Japanese and English. A previously proposed post-ordering method for Japanese-to-English translation first translates a Japanese sentence ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

April 2020

10688 pages

ISBN:9781450367080

DOI:10.1145/3313831

General Chairs:
Regina Bernhaupt
Eindhoven University of Technology, Netherlands
,
Florian 'Floyd' Mueller
Monash University, Australia
,
David Verweij
Newcastle University, UK
,
Josh Andres
RMIT, Australia
,
Program Chairs:
Joanna McGrenere
University of British Columbia, Canada
,
Andy Cockburn
University of Canterbury, New Zealand
,
Ignacio Avellino
University of Maryland Baltimore County, USA
,
Alix Goguey
Grenoble Alpes University, France
,
Pernille Bjørn
University of Copenhagen, Denmark
,
Shengdong (Shen) Zhao
National University of Singapore, Singapore
,
Briane Paul Samson
Future University Hakodate, Japan & De La Salle University, Philippines
,
Rafal Kocielnik
University of Washington, USA

Copyright © 2020 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 April 2020

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '20

Sponsor:

SIGCHI

CHI '20: CHI Conference on Human Factors in Computing Systems

April 25 - 30, 2020

HI, Honolulu, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
4,308
Total Downloads

Downloads (Last 12 months)3,016
Downloads (Last 6 weeks)348

Reflects downloads up to 19 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Adjagbodjou AKaufman G(2024)Envisioning Support-Centered Technologies for Language Practice and Use: Needs and Design Opportunities for Immigrant English Language Learners (ELLs)Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642236(1-15)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642236
Calambur VJun DSchiaffino MZhang ZHuh-Yoo J(2024)A case for "little English" in Nurse Notes from the Telehealth Intervention Program for Seniors: Implications for Future Design and ResearchProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641961(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641961
Lee NLee K(2024)Travelers’ viewpoints on machine translation using Q methodology: a perspective of consumption value theoryInformation Technology & Tourism10.1007/s40558-024-00296-xOnline publication date: 13-Aug-2024
https://doi.org/10.1007/s40558-024-00296-x
Hudelson PChappuis F(2024)Using Voice-to-Voice Machine Translation to Overcome Language Barriers in Clinical Communication: An Exploratory StudyJournal of General Internal Medicine10.1007/s11606-024-08641-w39:7(1095-1102)Online publication date: 12-Feb-2024
https://doi.org/10.1007/s11606-024-08641-w
Sitbon LBalasuriya SPercy L(2023)Guiding non-verbal conversation about meal choices with intelligent technologiesProceedings of the 35th Australian Computer-Human Interaction Conference10.1145/3638380.3638443(350-359)Online publication date: 2-Dec-2023
https://dl.acm.org/doi/10.1145/3638380.3638443
Tseng YJarupreechachan WLee T(2023)Understanding the Benefits and Design of Chatbots to Meet the Healthcare Needs of Migrant WorkersProceedings of the ACM on Human-Computer Interaction10.1145/36101067:CSCW2(1-34)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3610106
Heuer HGlassman E(2023)Accessible Text Tools for People with Cognitive Impairments and Non-Native Readers: Challenges and OpportunitiesProceedings of Mensch und Computer 202310.1145/3603555.3603569(250-266)Online publication date: 3-Sep-2023
https://dl.acm.org/doi/10.1145/3603555.3603569
Heuer HGlassman E(2023)Accessible Text Tools: Where They Are Needed & What They Should Look LikeExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585749(1-7)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3585749
Robertson SWang ZMoritz DKery MHohman F(2023)Angler: Helping Machine Translation Practitioners Prioritize Model ImprovementsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580790(1-20)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580790
Rozmysłowicz T(2023)The politics of machine translation. Reprogramming translation studiesPerspectives10.1080/0907676X.2023.229257132:3(493-507)Online publication date: 11-Dec-2023
https://doi.org/10.1080/0907676X.2023.2292571
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents