Multimodal dialogue processing for machine translation
July 2019
Pages 577 - 620
References
[1]
D. Bahdanau, K. Cho, and Y. Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of the International Conference on Learning Representations (ICLR). 584, 586
[2]
H. U. Block, St. Schachtl, and M. Gehrke. 2000. Adapting a large scale MT system for Spoken Language. In W. Wahlster, editor, Verbmobil: Foundations of Speech-to-Speech Translation, pp. 394--410. Springer Published, Berlin/Heidelberg, Germany. 606
[3]
H. Bourlard and N. Morgan. 1994. Connectionist Speech Recognition, A Hybrid Approach. Kluwer Academic Publishers.
[4]
H. Bourlard and Ch. Wellekens. 1989. Speech pattern discrimination and multilayer perceptrons. In Computer Speech and Language, (3), pp. 1--19. 587
[5]
P. F. Brown, S. A. Della Pietra, V. J. Della Pietra, and R. L. Mercer. 1993. The Mathematics of Statistical Machine Translation: Parameter Estimation. Computational Linguistics, 19(2): S. 263--311. 590
[6]
E. Cho, J. Niehues, T. L. Ha, M. Sperber, M. Mediani, and A. Waibel. 2016. Adaptation and Combination of NMT Systems: The KIT Translation Systems for IWSLT 2016. In Proceedings of the 13th International Workshop on Spoken Language Translation, IWSLT. Seattle. 588
[7]
E. Cho, J. Niehues, and A. Waibel. 2014a. Tight integration of speech disfluency removal into SMT. EACL, 2014, 43. Gothenburg, Sweden. 585
[8]
E. Cho, J. Niehues, and A. Waibel. 2014b. Machine Translation of Multi-party Meetings: Segmentation and Disfluency Removal Strategies. IWSLT. Lake Tahoe, US. 601
[9]
E. Cho, C. Fügen T. Herrmann, K. Kilgour, M. Mediani, C. Mohr, J. Niehues, K. Rottmann K. Saam, S. Stüker, and A. Waibel. 2013. A Real-World System for Simultaneous Translation of German Lectures. In Proceedings of the 14th Annual Conference of the International Speech Communication Association (Interspeech 2013). Lyon, France. 596, 598
[10]
K. Cho, B. van Merrienboer, Ç, Gülçehre, D. Bahdanau, F. Bougares, H. Schwenk. 2014. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, pp. 1724--1734. Doha, Qatar. 600
[11]
M. Eck, I. Lane, Y. Zhang, and A. Waibel. 2010 Jibbigo: Speech-to-Speech translation on mobile devices. In 2010 IEEE Spoken Language Technology Workshop, 165--166. 592
[12]
The Economist. June 12, 2006. How to build a Bablefish. In The Economist.
[13]
Ectaco. 1989. Ectaco eBook Readers and Translators. http://www.ectaco.com. 592
[14]
C. Fügen, A. Waibel, and M. Kolss. 2007. Simultaneous Translation of Lectures and Speeches. In Journal Machine Translation, 21(4), 209--252. 597
[15]
A. von Greve-Dierfeld. 2012. Uni-Übersetzungs-Automat: Don't worry about make. Spiegel-Online. http://www.spiegel.de/unispiegel/studium/dolmetscher-fuer-die-vorlesung-kit-entwickelt-uebersetzungsprogramm-a-838409.html 598
[16]
T.-L. Ha, J. Niehues, and A. Waibel. December 2016. Toward Multilingual Neural Machine Translation with Universal Encoder and Decoder. In Proceedings of the 13th International Workshop on Spoken Language Translation (IWSLT), 8--9. Seattle, WA. 586
[17]
Handelsblatt: Übersetzung. July 1991. Handelsblatt. http://isl.anthropomatik.kit.edu/cmu-kit/downloads/1991.07.30_Handelsblatt.pdf. 588
[18]
K. Hatazaki, J. Noguohi, A. Okumura, K. Yoshida, and T. Watanabe T. 1992. INTERTALKER: an experimental automatic interpretation system using conceptual representation. In Second International Conference on Spoken Language Processing. 588
[19]
S. Hourin, J. Binder, D. Yeager, P. Gamerdinger, K. Wilson, and K. Torres-Smith. 2013. Speech-to-Speech Translation Tool Limited Utility Assessment Report. Report OMB No.0704-0188. 593
[20]
S. Jaeger, S. Manke, J. Reichert, and A. Waibel. 2001. Online handwriting recognition: the NPen++ recognizer. International Journal on Document Analysis and Recognition, 3(3): 169--180. 609
[21]
S. Jaeger, S. Manke, and A. Waibel. September 2000. NPen++: An On-line Handwriting Recognition System. In Proceedings of the 7th International Workshop on Frontiers in Handwriting Recognition, IWFHR 2000, Amsterdam, The Netherlands, 11--13. 609
[22]
A. Jain and A. Waibel. August 1989. A Connectionist Parser Aimed at Spoken Language. In Proceedings of the 1st International Workshop on Parsing Technologies, IWPT 1989, Pittsburgh, PA. 28--31. 585
[23]
M. Johnson, M. Schuster, Q.V. Le, M. Krikun, Y. Wu, Z. Chen, N. Thorat, F. Viégas, M. Wattenberg, G. Corrado, M. Hughes, and J. Dean. 2017. Google's multilingual neural machine translation system: enabling zero-shot translation. Transactions of the Association for Computational Linguistics, vol. 5, pp. 339--351. 586
[24]
E. C. Kaiser. 2006. Using redundant speech and handwriting for learning new vocabulary and understanding abbreviations. In Proceedings of the 8th ACM International Conference on Multimodal Interfaces, pp. 347--356. ACM Press. 607
[25]
E. C. Kaiser. 2005. Multimodal new vocabulary recognition through speech and handwriting in a white-board scheduling application. In ACM Intelligent User Interfaces Conference, IUI '05, pp. 51--58. ACM Press, New York. 607
[26]
N. Kalchbrenner and P. Blunsom. 2013. Recurrent continuous translation models. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1700--1709. 584
[27]
P. Koehn, Y. Zhang, C. Dugast, J. Gauthier, S. Grimsey, S. Fünfer, M. Mueller, S. Stueker, and V. Steinbiss. EU-BRIDGE D6.3 Final Evaluation Report, (www.eu-bridge.eu). 602
[28]
P. Koehn, H. Hoang, A. Birch, C. Callison-Burch, M. Federico, N. Bertoldi, B. Cowan, W. Shen, C. Moran, R. Zens, C. Dyer, O. Bojar, A. Constantin, and E. Herbst. 2007. Moses: Open Source Toolkit for Statistical Machine Translation. ACL, Prague, Czech Republic. 590
[29]
P. Koehn, and K. Knight. 2003. Empirical Methods for Compound Splitting. EACL, pp. 187--193. Budapest, Hungary. 599
[30]
R. Kumar, S. Hewavitharana, N. Zinovieva, M. E. Roy, and E. Pattison-Gordon. 2015. Error-Tolerant Speech- to-Speech Translation. In Proceedings of MT Summit XV, Volume 1: MT Researchers' Track, pp. 229--239. Miami, FL. 606
[31]
A. Lavie, A. Waibel, L. Levin, M. Finke, D. Gates, M. Gavalda, T. Zeppenfeld, and P. Zhan. 1997. JANUS III: Speech-to-Speech Translation in Multiple Languages. International Conferences on Acoustics, Speech, and Signal Processing, ICASSP. Munich, Germany. 590
[32]
W. Lewis. November 2015. Skype Translator: Breaking Down Language and Hearing Barriers. AsLing's 37th Translating and the Computer Conference, 27--28. London. 595
[33]
L. Maier-Hein, F. Metze, T. Schultz, and A. Waibel. November 2005. Session Independent non-audible speech recognition using surface ellectromyography. In Proceedings of ASRU, Cancun, Mexico. 610
[34]
S. Manke, M. Finke, and A. Waibel. 1995. The use of dynamic writing information in a connectionist on-line cursive handwriting recognition system. Advances in Neural Information Processing Systems, 1093--1100. 609
[35]
Y. Miao, M. Gowayyed, and F. Metze. 2015. EESEN: End-to-End Speech Recognition using Deep RNN Models and WFST-Based Decoding. Automatic Speech Recognition and Understanding Workshop (ASRU), Scottsdale, AZ. 588, 607
[36]
R. Miikkulainen and M. D. Dyer. 1989. A modular neural network architecture for sequential paraphrasing of script-based stories. In Proceedings of the International Joint Conference on Neural Networks. IEEE. 585
[37]
T. Morimoto, Takezawa, F. Yato, S. Sagayama, T. Tashiro, M. Nagata, and A. Kurematsu. 1993. ATR's speech translation system: ASURA. In Proceedings Eurospeech '93, pp. 1291--1294. Geneva, Italy. 588, 590
[38]
G. Neubig. 2016. Lexicons and Minimum Risk Training for Neural Machine Translation: NAIST-CMU at WAT2016. In Proceedings of the 3rd Workshop on Asian Translation (WAT). 588
[39]
J. Niehues and A. Waibel. 2011. Using Wikipedia to Translate Domain-specific Terms in SMT. In Proceedings of the Eight International Workshop on Spoken Language Translation (IWSLT). 600
[40]
F. J. Och, and H. Ney. 2004. The alignment template approach to statistical machine translation. In Journal Computational Linguistics, 30(4): pp. 417--449. 590
[41]
S. Oviatt and R. VanGent. 1996. Error resolution during multimodal human-computer interaction. In Proceedings of the Fourth International Conference on Spoken Language Processing, Vol. 1, pp. 204--207.
[42]
S. Oviatt and P. R. Cohen. 1992. Spoken Language in interpreted telephone dialogues. In Computer Speech and Language, 6(3): pp. 277--302. 580
[43]
D. B. Roe. 1992. A spoken language translator for restricted-domain context-free languages. In Speech Communication, 11(2--3: pp. 311--319. 588
[44]
S. Stüker, M. Federico, Ph., Koehn, H. Ney, M. Rödder, M. Simpson, V. Steinbiss, and A. Tescari. 2015. EU-BRIDGE Final Report. www.eu-bridge.eu 603
[45]
S. Stüker, C. Zong, J. Reichert W. Cao, M. Kolss, G. Xie, K. Peterson, P. Ding, V. Arranz, J. Yu and A. Waibel. 2006. Speech-to-Speech Translation Services for the Olympic Games 2008, 3rd Joint Workshop on Multimodal Interaction and Related Machine Learning Algorithms, MLMI 2006, Washington D.C. 592
[46]
M. Seligman, A. Waibel, and A. Joscelyne. 2017. TAUS Speech-to-Speech Translation Technology Report. TAUS Report.
[47]
R. Sennrich, B. Haddow, and A. Birch. 2016. Edinburgh Neural Machine Translation Systems for WMT 16. In Proceedings of the First Conference on Machine Translation (WMT16). Berlin, Germany. 588
[48]
T. Starner, J. Makhoul, R. Schwartz, and G. Chou. 1994. On-line cursive handwriting recognition using speech recognition methods. Acoustics, Speech, and Signal Processing, ICASSP. 609
[49]
B. Suhm, B. Myers, and A. Waibel. May 1999. Model-based And Empirical Evaluation Of Multimodal Interactive Error Correction. Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, CHI 1999, Pittsburgh, PA. 606
[50]
B. Suhm, B. Myers, and A. Waibel. 1996. Interactive recovery from speech recognition errors in speech user interfaces. In Proceedings of the International Conference on Spoken Language Processing, pp. 861--864. Philadelphia, PA. 606
[51]
I. Sutskever, O. Vinyals, and Q. V. Le. 2014. Sequence to Sequence Learning with Neural Networks. In Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014 pp. 3104--3112. Quebec, Canada. 584, 585
[52]
T. Takezawa, T. Morimoto, Y. Sagisaka, N. Campbell, H. Iida, F. Sugaya, A. Yokoo, and S. Yamamoto. 1998. A Japanese-to-English Speech Translation System: ATR-MATRIX. In Proceedings ICSLP'98, pp. 779--782. Sydney, Australia. 590
[53]
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, l. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. 2017. Attention Is All You Need. CoRR, abs/1706.03762. 586
[54]
Voxtec, Advancing Voice Technology<sup>™</sup>. http://www.voxtec.com. 592
[55]
W. Wahlster. 1993. Verbmobil---Translation of Face-To-Face Dialogues. In Grundlagen und Anwendungen der Künstlichen Intelligenz. Springer-Verlag, Berlin Heidelberg. 588
[56]
A. Waibel. 2002. Portable object identification and translation system. US Patent App. 10/090,559. 608
[57]
A. Waibel. 2015a. Translation Training with Cross-Lingual Multimedia Support, 2018, US Patent 9,892,115 B2; CIP Application #14/589,658, filed 2015.
[58]
A. Waibel. 2015b. Translation and Integration of Presentation Materials with Cross-Lingual Multimedia Support, 2017 US Patent 9,678,953 B2, 2017; CIP Application #14/589,653, filed 2015
[59]
A. Waibel. 2014. Translation and Integration of Presentation Materials with Cross-Lingual Multimedia Support, 2014; Patent Pub. #US 2014/0365202 A1; Appl. # 14/302,146, filed 2014
[60]
A. Waibel and I. R. Lane. 2015. Enhanced speech-to-speech translation system and methods for adding a new word, US Patent. 607
[61]
A. Waibel and R. Stiefelhagen, editors. 2009. Computers in the Human Interaction Loop. Springer, London. 610, 611
[62]
A. Waibel and A. McNair. 1998b. Locating and correcting erroneously recognized portions of utterances by rescoring based on two n-best lists, US Patent 5,712,957b. 606
[63]
A. Waibel, B. Suhm, and A. McNair. 1998a. Method and apparatus for correcting and repairing machine-transcribed input using independent or cross-modal secondary input. US Patent 5,855,000. 606
[64]
A. Waibel, A. Jain, A. McNair, H. Saito, A. Hauptmann, and J. Tebelskis. May 1991. JANUS: A Speech-to-Speech Translation System Using Connectionist and Symbolic Processing Strategies. In Proceedings of the International Conference on Acoustics, Speech and Signal Processing, Toronto. 588, 606
[65]
A. Waibel, A. Hanazawa, G. Hinton, K. Shikano, and K. Lang. March 1989. Phoneme Recognition Using Time-Delay Neural Networks. In IEEE Transactions of the Acoustics, Speech and Signals Processing Society, 347(3). 587
[66]
A. Waibel. March 1989. Modular Construction of Time-Delay Neural Networks for Speech Recognition. In Journal for Neural Computation, MIT Press Journals, 1. 587
[67]
Y.-Y. Wang and A. Waibel. July 1997. Decoding Algorithm In Statistical Machine Translation. In Proceedings of the 35th Annual Meeting of the ACL joint with the 8th Meeting of the European Chapter of the ACL 1997, ACL/EACL 1997, Madrid, Spain. 590
[68]
Y.-Y. Wang and A. Waibel A. May 1991. A Connectionist Model for Dialog Processing. In Proceedings of the International Conference on Acoustics, Speech and Signal Processing, ICASSP 1991, Toronto, Canada. 585
[69]
J. Yang, J. Gao, Y. Zhang, X. Chen, and A. Waibel. 2001. An Automatic Sign Recognition and Translation System. Workshop on Perceptual User Interfaces 2001, PUI 2001. 608
[70]
T. Zenkel, R. Sanabria, F. Metze, J. Niehues, M. Sperber, S. Stüker, and A. Waibel. 2017. Comparison of Decoding Strategies for CTC Acoustic Models. In Proceedings of Interspeech, pp. 513--517. Stockholm, Sweden. 588, 607
[71]
Y. Zhang, B. Zhao, J. Yang, and A. Waibel. September 2002. Automatic SIGN Translation, 7th International Conference on Spoken Language Processing, 2nd Interspeech Event, ICSLP 2002 - Interspeech 2002, Denver, CO. 609
[72]
G. Zweig, C. Yu, K. Droppo, and A. Stolcke. March 2016. Advances in All-Neural Speech Recognition. The 41st IEEE International Conference on Acoustics, Speech and Signal Processing, Shanghai, China. 588, 607
Recommendations
From vocal to multimodal dialogue management
ICMI '06: Proceedings of the 8th international conference on Multimodal interfacesMultimodal, speech-enabled systems pose different research problems when compared to unimodal, voice-only dialogue systems. One of the important issues is the question of how a multimodal interface should look like in order to make the multimodal ...
Human-robot collaborative tutoring using multiparty multimodal spoken dialogue
HRI '14: Proceedings of the 2014 ACM/IEEE international conference on Human-robot interactionIn this paper, we describe a project that explores a novel experimental setup towards building a spoken, multi-modally rich, and human-like multiparty tutoring robot. A human-robot interaction setup is designed, and a human-human dialogue corpus is ...
Dialogue Machine Translation System Using Multiple Translation Processors
DEXA '00: Proceedings of the 11th International Workshop on Database and Expert Systems ApplicationsIn natural dialogues, speakers may make some kinds of mistakes or so called irregular expressions. One of the most difficult problems which Dialogue Machine Translations must deal with is translating irregular expressions in the natural conversation, ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
July 2019
813 pages
ISBN:9781970001754
DOI:10.1145/3233795
Publisher
Association for Computing Machinery and Morgan & Claypool
Publication History
Published: 01 July 2019
Check for updates
Qualifiers
- Research-article
Appears in
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 126Total Downloads
- Downloads (Last 12 months)9
- Downloads (Last 6 weeks)3
Reflects downloads up to 01 Oct 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in