short-paper

Multimodal Chinese text entry with speech and keypad on mobile devices

Authors:

Hongan WangAuthors Info & Claims

IUI '08: Proceedings of the 13th international conference on Intelligent user interfaces

Pages 341 - 344

https://doi.org/10.1145/1378773.1378825

Published: 13 January 2008 Publication History

Get Access

Abstract

Chinese text entry is challenging on mobile devices which rely on keypad input. Entering one character may require many key presses. This paper proposes a multimodal text entry technique for Chinese. In this method, Chinese user can enter Chinese text by simultaneously using the simplified phonemic input method named Jianpin with keypad and speech utterance. The key of the technique is a multimodal fusion algorithm, which synchronizes speech and keypad input and fuses redundant information from two modalities to get the best candidate. A preliminary evaluation shows that users appreciate this technique and it could reduce key presses and enhance the input efficiency.

References

[1]

Ao, X., Wang, X. G., Tian, F., Dai, G. Z. and Wang, H. A. Crossmodal error dorrection of continuous handwriting recognition by speech. In Proc. IUI 2007. ACM Press (2007), 243--250.

Digital Library

Google Scholar

[2]

Chen, Y., Chinese Language Processing, Shanghai Education publishing company, China, 1997.

Google Scholar

[3]

Hsu, B. J., Mahajan, M. and Acero, A. Multimodal Text Entry on Mobile Devices. IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) 2005

Google Scholar

[4]

Hsu, S. C. A flexible Chinese character input scheme. In Proc. UIST 1991. ACM Press (1991), 195--200.

Digital Library

Google Scholar

[5]

Kaiser, E. C. Using redundant speech and handwriting for learning new vocabulary and understanding abbreviations. In Proc ICMI 2006. ACM Press (2006), 347--356

Digital Library

Google Scholar

[6]

Kurihara, K., Goto, M., Ogata, J. and Igarashi, T. Speech Pen: Predictive Handwriting based on Ambient Multimodal Recognition. In Proc. CHI 2006. ACM Press (2006), 851--860.

Digital Library

Google Scholar

[7]

Lin, M., and Sears, A. Graphics matter: a case study of mobile phone keypad design for Chinese input. Ext. Abstracts 2005. ACM Press (2005), 1593--1596

Digital Library

Google Scholar

[8]

Oviatt, S. Ten Myths of Multimodal Interaction, Communications of the ACM, 42, 9(1999), 74--81

Digital Library

Google Scholar

[9]

T9. http://www.nuance.com/t9/textinput/

Google Scholar

[10]

Vertanen, K. Efficient Computer Interfaces Using Continuous Gestures, Language Models, and Speech. M. Phil Thesis, University of Cambridge, 2004

Google Scholar

[11]

Wang, J. T., Zhai, S. M. and Su, H. Chinese input with keyboard and eye-tracking -- an anatomical study. In Proc. CHI 2001. ACM Press (2001), 349--356.

Digital Library

Google Scholar

[12]

Wang, X. G., Li, J. F., Ao, X., Wang, G. and Dai, G. Z. Multimodal Error Correction for Continuous Handwriting Recognition in Pen-based user Interfaces. In Proc IUI 2006. ACM Press (2006), 324--326

Digital Library

Google Scholar

[13]

Zhai, S. M., Kristensson, P. O. and Smith, B. A. In search of effective text input interfaces for off the desktop computing. Interacting with Computers. 17, 3 (2005), 229--250.

Crossref

Google Scholar

Cited By

View all

Zhang DChou HZhou L(2012)Hype or Ready for Prime Time?International Journal of Handheld Computing Research10.4018/jhcr.20121001033:4(40-55)Online publication date: 1-Oct-2012
https://dl.acm.org/doi/10.4018/jhcr.2012100103
Dearman DKarlson AMeyers BBederson BMould DNoël S(2010)Multi-modal text entry and selection on a mobile deviceProceedings of Graphics Interface 201010.5555/1839214.1839219(19-26)Online publication date: 31-May-2010
https://dl.acm.org/doi/10.5555/1839214.1839219
Xue LSun X(2010)Natural behavioral patterns of speech recognition error recovery2010 Sixth International Conference on Natural Computation10.1109/ICNC.2010.5582480(2135-2138)Online publication date: Aug-2010
https://doi.org/10.1109/ICNC.2010.5582480

Index Terms

Multimodal Chinese text entry with speech and keypad on mobile devices
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Predicting Chinese text entry speeds on mobile phones
CHI '10: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Chinese text entry on mobile phones is critical considering the large number of Chinese speakers worldwide and as a key task in many core applications. But there is still a lack of both empirical data and predictive models that explore the pattern of ...
Improved word list ordering for text entry on ambiguous keypads
NordiCHI '08: Proceedings of the 5th Nordic conference on Human-computer interaction: building bridges

We present a design methodology for small ambiguous keypads, where input often produces a list of candidate words for a given desired word. The methodology uses context through semantic relatedness and a part-of-speech language model to improve the ...
Speech recognition on mobile devices
Mobile Multimedia Processing

The enthusiasm of deploying automatic speech recognition (ASR) on mobile devices is driven both by remarkable advances in ASR technology and by the demand for efficient user interfaces on such devices as mobile phones and personal digital assistants (...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

IUI '08: Proceedings of the 13th international conference on Intelligent user interfaces

January 2008

458 pages

ISBN:9781595939876

DOI:10.1145/1378773

General Chair:
Steffen Staab
Universitat Koblenz-Landau, Germany

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 January 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

Conference

IUI08

Sponsor:

IUI08: 13th International Conference on Intelligent User Interfaces

January 13 - 16, 2008

Gran Canaria, Spain

Acceptance Rates

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Sponsor:
sigai
sigai

30th International Conference on Intelligent User Interfaces

March 24 - 27, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
301
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 19 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Zhang DChou HZhou L(2012)Hype or Ready for Prime Time?International Journal of Handheld Computing Research10.4018/jhcr.20121001033:4(40-55)Online publication date: 1-Oct-2012
https://dl.acm.org/doi/10.4018/jhcr.2012100103
Dearman DKarlson AMeyers BBederson BMould DNoël S(2010)Multi-modal text entry and selection on a mobile deviceProceedings of Graphics Interface 201010.5555/1839214.1839219(19-26)Online publication date: 31-May-2010
https://dl.acm.org/doi/10.5555/1839214.1839219
Xue LSun X(2010)Natural behavioral patterns of speech recognition error recovery2010 Sixth International Conference on Natural Computation10.1109/ICNC.2010.5582480(2135-2138)Online publication date: Aug-2010
https://doi.org/10.1109/ICNC.2010.5582480

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Predicting Chinese text entry speeds on mobile phones

Improved word list ordering for text entry on ambiguous keypads

Speech recognition on mobile devices