Abstract
Transliteration is crucial in natural language processing as it enables the conversion between two languages while retaining their phonetic representation. The ability to transliterate the Bengali language is essential for cross-lingual communication. Most Bengali machine transliteration techniques rely on one-hot coding to numerically represent features, which is computationally intensive, requires significant memory, and results in lower accuracy in some cases. Addressing these limitations, we present an efficient feature representation method utilizing binary coding and compare it against the one-hot coding approach using two machine learning models: SVM and random forest. The proposed method is evaluated on the Dakshina dataset and the NEWS 2018 dataset for Bengali to English transliteration. The experimental results indicate that our approach outperforms the traditional methods while requiring significantly less memory and training time. Overall, the proposed method enables a better romanization of frequently used words.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Akan MF (2018) Transliteration and translation from Bangla into English: a problem-solving approach. British J English Linguist 6(6):1–21
Anwarus Salam KM, Yamada S, Nishino T (2013) How to translate unknown words for English to Bangla machine translation using transliteration. J Comput 8. https://doi.org/10.4304/cp.8.5.1167-1174
Arbabi M, Fischthal SM, Cheng VC, Bart EC, Bart E (1994) Algorithms for Arabic name transliteration. IBM J Res Develop 38(2):183–194. https://doi.org/10.1147/rd.382.0183
Basu J, Basu T, Mitra M, Mandal SKD (2009) Grapheme to phoneme (g2p) conversion for bangla. In: 2009 oriental COCOSDA international conference on speech database and assessments, pp. 66–71. https://doi.org/10.1109/ICSDA.2009.5278373
Chen N, Banchs RE, Zhang M, Duan X, Li H (2018) Report of NEWS 2018 named entity transliteration shared task. In: Proceedings of the seventh named entities workshop. Association for Computational Linguistics, Melbourne, Australia, pp 55–73. https://doi.org/10.18653/v1/W18-2409
Dasgupta T, Sinha M, Anupam B (2015) Resource creation and development of an english-bangla back transliteration system. Int J Knowl-Based Intell Eng Syst 19:35–46. https://doi.org/10.3233/KES-150307
Ekbal A, Naskar SK, Bandyopadhyay S (2006) A modified joint source-channel model for transliteration. In: Proceedings of the COLING/ACL 2006 main conference poster sessions. Association for Computational Linguistics, Sydney, Australia, pp 191–198. https://aclanthology.org/P06-2025
Goutte C, Gaussier E (2005) A probabilistic interpretation of precision, recall and f-score, with implication for evaluation. In: Losada DE, Fernández-Luna JM (eds) Advances in Information Retrieval, vol 3408. Springer, Berlin Heidelberg, Berlin, Heidelberg, pp 345–359
Haizhou L, Min Z, Jian S (2004) A joint source-channel model for machine transliteration. In: Proceedings of the 42nd annual meeting on association for computational linguistics, ACL ’04. Association for Computational Linguistics, USA, pp 159–166. https://doi.org/10.3115/1218955.1218976
Karimi S, Scholer F, Turpin A (2011) Machine transliteration survey. ACM Comput Surv 43(3). https://doi.org/10.1145/1922649.1922654
Kaur V, Sarao AK, Singh J (2014) Hybrid approach for hindi to english transliteration system for proper nouns. Int J Comput Sci Inf Technol (IJCSIT) 5(5):6361–6366
Khan FS, Mushabbir MA, Irbaz MS, Nasim MAA (2021) End-to-end natural language understanding pipeline for bangla conversational agents. In: 2021 20th IEEE International conference on machine learning and applications (ICMLA), pp 205–210. https://doi.org/10.1109/ICMLA52953.2021.00039
Kundu S, Paul S, Pal S (2018) A deep learning based approach to transliteration. In: Proceedings of the seventh named entities workshop. Association for Computational Linguistics, Melbourne, Australia, pp 79–83. https://doi.org/10.18653/v1/W18-2411. https://aclanthology.org/W18-2411
Oh JH, Choi KS (2006) An ensemble of transliteration models for information retrieval. Inf Process Manage 42(4):980–1002. https://doi.org/10.1016/j.ipm.2005.09.007
Potdar K, Pardawala TS, Pai CD (2017) A comparative study of categorical variable encoding techniques for neural network classifiers. Int J Comput Appl 175(4):7–9. https://doi.org/10.5120/ijca2017915495
Rathod P, Dhore M, Dhore R (2013) Hindi and Marathi to English machine transliteration using SVM. Int J Nat Lang Comput (IJNLC) 2:55–71. https://doi.org/10.5121/ijnlc.2013.2404
Riyadh RR, Kondrak G (2019) Joint approach to deromanization of code-mixed texts. In: Proceedings of the sixth workshop on nlp for similar languages, varieties and dialects. Association for Computational Linguistics, Ann Arbor, Michigan, pp. 26–34. https://doi.org/10.18653/v1/W19-1403
Rizvee RA, Mahmood A, Mullick SS, Hakim S (2022) A robust three-stage hybrid framework for english to bangla transliteration. Int J Nat Lang Comput 11(1):15. https://doi.org/10.5121/ijnlc.2022.11104
Roark B, Wolf-Sonkin L, Kirov C, Mielke SJ, Johny C, Demirsahin I, Hall K (2020) Processing South Asian languages written in the Latin script: the Dakshina dataset. In: Proceedings of the twelfth language resources and evaluation conference. European Language Resources Association, Marseille, France, pp 2413–2423. https://aclanthology.org/2020.lrec-1.294
Sarkar K, Chatterjee S (2017) Bengali-to-english forward and backward machine transliteration using support vector machines. In: Mandal JK, Dutta P, Mukhopadhyay S (eds) Computational intelligence, communications, and business analytics. Springer Singapore, Singapore, pp 552–566
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Tipu, A.D., Fahad, M., Mandal, A.K. (2024). A Romanization Method for the Bengali Language with Efficient Encoding Scheme. In: Arefin, M.S., Kaiser, M.S., Bhuiyan, T., Dey, N., Mahmud, M. (eds) Proceedings of the 2nd International Conference on Big Data, IoT and Machine Learning. BIM 2023. Lecture Notes in Networks and Systems, vol 867. Springer, Singapore. https://doi.org/10.1007/978-981-99-8937-9_41
Download citation
DOI: https://doi.org/10.1007/978-981-99-8937-9_41
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8936-2
Online ISBN: 978-981-99-8937-9
eBook Packages: EngineeringEngineering (R0)