Abstract
Cyrillic and Traditional Mongolian are the two main members of the Mongolian writing system. The Cyrillic-Traditional Mongolian Bidirectional Conversion (CTMBC) task includes two conversion processes, including Cyrillic Mongolian to Traditional Mongolian (C2T) and Traditional Mongolian to Cyrillic Mongolian conversions (T2C). Previous researchers adopted the traditional joint sequence model, since the CTMBC task is a natural Sequence-to-Sequence (Seq2Seq) modeling problem. Recent studies have shown that Recurrent Neural Network (RNN) and Self-attention (or Transformer) based encoder-decoder models have shown significant improvement in machine translation tasks between some major languages, such as Mandarin, English, French, etc. However, an open problem remains as to whether the CTMBC quality can be improved by utilizing the RNN and Transformer models. To answer this question, this paper investigates the utility of these two powerful techniques for CTMBC task combined with agglutinative characteristics of Mongolian language. We build the encoder-decoder based CTMBC model based on RNN and Transformer respectively and compare the different network configurations deeply. The experimental results show that both RNN and Transformer models outperform the traditional joint sequence model, where the Transformer achieves the best performance. Compared with the joint sequence baseline, the word error rate (WER) of the Transformer for C2T and T2C decreased by 5.72% and 5.06% respectively.
This research is funded by the National Key Research and Development Program of China (No. 2018YFE0122900), China National Natural Science Foundation (No. 62066033), the High-level Talents Introduction Project of Inner Mongolia University (No. 10000-22311201/002) and the Young Scientists Fund of the National Natural Science Foundation of China (No. 62206136), Applied Technology Research and Development Program of Inner Mongolia Autonomous Region (No. 2019GG372, 2020GG0046, 2021GG0158, 2020PT002), Young science and technology talents cultivation project of Inner Mongolia University (No. 21221505), The Research Program of The National Social Science Fund of China (No. 18XYY030).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Chinggaltai: A Grammar of the Mongolian Language. Inner Mongolia Peoples Publishing House, Hohhot (1991)
Chuma, S.: A Comparative Study of Mongolian and Cyrillic Orthography. Inner Mongolia Education Press, Hohhot (2010)
Bao, F., Gao, G., Yan, X., Wei, H.: Research on conversion approach between traditional Mongolian and Cyrillic Mongolian. Comput. Eng. Appl. 206–211 (2014)
Li, H., Sarina, B.: The study of comparison and conversion about traditional Mongolian and Cyrillic Mongolian. In: 2011 4th International Conference on Intelligent Networks and Intelligent Systems, pp. 199–202. IEEE (2011)
Sarina, B.: The research on conversion of noun and its case from Classic Mongolian into Cyrillic Mongolian. Master’s thesis, Inner Mongolia University (2009)
Gao, H., Ma, x.: Automatic system on Cyrillic Mongolian webpage conversion to traditional Mongolian script. J. Inner Mongolia Univ. Nationalities 18(5), 17–18 (2012)
Bao, F., Gao, G., Wang, H., Lu, M.: Combining of rules and statistics for Cyrillic Mongolian to traditional Mongolian conversion. J. Chin. Inf. Procession 31(3), 156–162 (2017)
Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. Comput. Sci. (2014)
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems, vol. 27 (2014)
Kalchbrenner, N., Blunsom, P.: Recurrent continuous translation models. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1700–1709 (2013)
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. Comput. Sci. (2014)
Wu, Y., et al.: Google’s neural machine translation system: bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144 (2016)
Liu, R., Bao, F., Gao, G., Zhang, H., Wang, Y.: Improving Mongolian phrase break prediction by using syllable and morphological embeddings with BiLSTM model. In: Interspeech, pp. 57–61 (2018)
Liu, R., Sisman, B., Bao, F., Gao, G., Li, H.: Modeling prosodic phrasing with multi-task learning in Tacotron-based TTS. IEEE Sig. Process. Lett. 27, 1470–1474 (2020)
Liu, R., Sisman, B., Bao, F., Yang, J., Gao, G., Li, H.: Exploiting morphological and phonological features to improve prosodic phrasing for Mongolian speech synthesis. IEEE/ACM Trans. Audio Speech Lang. Process. 29, 274–285 (2020)
Liu, R., Sisman, B., Li, J., Bao, F., Gao, G., Li, H.: Teacher-student training for robust Tacotron-based TTS. In: ICASSP 2020–2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6274–6278. IEEE (2020)
Wang, Y., et al.: Tacotron: towards end-to-end speech synthesis. arXiv preprint arXiv:1703.10135 (2017)
Bisani, M., Ney, H.: Joint-sequence models for grapheme-to-phoneme conversion. Speech Commun. 50(5), 434–451 (2008)
Wang, Y., Bao, F., Zhang, H., Gao, G.: Joint alignment learning-attention based model for grapheme-to-phoneme conversion. In: ICASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 7788–7792. IEEE (2021)
Yao, K., Zweig, G.: Sequence-to-sequence neural net models for grapheme-to-phoneme conversion. Comput. Sci. (2015)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding, pp. 4171–4186. Association for Computational Linguistics, June 2019. https://doi.org/10.18653/v1/N19-1423, https://aclanthology.org/N19-1423
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R.R., Le, Q.V.: Xlnet: generalized autoregressive pretraining for language understanding. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
Cho, K., Van Merriënboer, B., Bahdanau, D., Bengio, Y.: On the properties of neural machine translation: encoder-decoder approaches. Comput. Sci. (2014)
Lu, M., Bao, F., Gao, G.: Language model for Mongolian polyphone proofreading. In: Sun, M., Wang, X., Chang, B., Xiong, D. (eds.) CCL/NLP-NABD -2017. LNCS (LNAI), vol. 10565, pp. 461–471. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-69005-6_38
Uganbater, D.: Research on Cyrillic and Mongolian script’s morphlolgy and conversion system. Ph.D. thesis, Inner Mongolia University (2014)
Dulamragchaa, U., Chadraabal, S., Ivanov, B., Baatarkhuu, M.: Mongolian language morphology and its database structure. In: 2017 International Conference on Green Informatics (ICGI), pp. 282–285. IEEE (2017)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Zhang, Z.: New Mongolian Chinese Dictionary. Commercial Press, Beijing (2011)
Vaswani, A., et al.: Tensor2tensor for neural machine translation. arXiv preprint arXiv:1803.07416 (2018)
Popel, M., Bojar, O.: Training tips for the transformer model. arXiv preprint arXiv:1804.00247 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Na, M., Liu, R., Bao, F., Gao, G. (2023). A Deep Investigation of RNN and Self-attention for the Cyrillic-Traditional Mongolian Bidirectional Conversion. In: Tanveer, M., Agarwal, S., Ozawa, S., Ekbal, A., Jatowt, A. (eds) Neural Information Processing. ICONIP 2022. Communications in Computer and Information Science, vol 1793. Springer, Singapore. https://doi.org/10.1007/978-981-99-1645-0_51
Download citation
DOI: https://doi.org/10.1007/978-981-99-1645-0_51
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-1644-3
Online ISBN: 978-981-99-1645-0
eBook Packages: Computer ScienceComputer Science (R0)