Abstract
The globalized information-sharing phenomena facilitated by technologies such as the Internet have increased the demand for translation services. Automating translation has been at the forefront of solutions to address the demand. Automatic translation services have been available for sometimes provided by tech companies such as Google; however, achieving full translation accuracy is an ongoing challenge. In this paper, a fuzzy logic-based evaluation metric is proposed for evaluating machine translation accuracy. Evaluation results generated by the metric is compared with evaluation results generated by the bilingual evaluation understudy (BLEU) which is one of the most widely used machine translation accuracy evaluation metrics. The accuracy of evaluation results produced by both metrics are benchmarked against human-based translation accuracy evaluations for over a set of sentences translated from Turkish to English by tools Google translation, Yandex translation, and a simple neural machine translation prototype developed by the authors. The results show that the proposed fuzzy logic-based metric evaluates the accuracy of machine translations more effectively than the BLEU metric.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Novikova J, Dušek O, Curry AC, Rieser V (2017) Why we need new evaluation metrics for NLG. arXiv preprint arXiv:1707.06875
Malik P, Baghel AS (2018) A summary and comparative study of different metrics for machine translation evaluation. In: 2018 8th International conference on cloud computing, data science and engineering (Confluence), pp 55–60. https://doi.org/10.1109/CONFLUENCE.2018.8442777
Reiter E (2018) A structured review of the validity of BLEU. Comput Linguist 44(3):393–401. https://doi.org/10.1162/coli_a_00322
Roseline P, Ganesan N, Tauro C (2015) A study of applications of fuzzy logic in various domains of agricultural sciences, In: International conference on current trends in advanced computing (ICCTAC-2015), vol 1, pp 15–18
Gürsel G et al (2016) Healthcare, uncertainty, and fuzzy logic. Digital Med 2(3):101. https://doi.org/10.4103/2226-8561.194697
Gupta P (2017) Applications of fuzzy logic in daily life. Int J Adv Res Comput Sci 8(5)
Jing X, Zhang Y, Hu Q, Rayz JT (2021) Modeling fuzzy cluster transitions for topic tracing. arXiv preprint arXiv:2104.08258
Makkar R (2018) Application of fuzzy logic: a literature review. Int J Statist Appl Math 3(1):357–359
Truck I, Abchir M-A (2017) Natural language processing and fuzzy tools for business processes in a geolocation context. Adv Artif Intell 2017. https://doi.org/10.1155/2017/9462457
Gupta C, Jain A, Joshi N (2018) Fuzzy logic in natural language processing—a closer view. Procedia Comput. Sci. 132:1375–1384. https://doi.org/10.1016/j.procs.2018.05.052
Masum AKM, Khandaker MAI, Alam GR (2011) A new approach for semantic web searching using fuzzy logic and natural language processing. J Comput 3(6)
Rana M, Atique M (2016) Use of Fuzzy tool for example-based machine translation. Procedia Comput Sci 79:199–206. https://doi.org/10.1016/j.procs.2016.03.026
Carvalho JP, Batista F, Coheur L (2012) A critical survey on the use of fuzzy sets in speech and natural language processing. In: 2012 IEEE international conference on fuzzy systems, pp 1–8. https://doi.org/10.1109/FUZZ-IEEE.2012.6250803
Popescu VF, Pistol MS (2021) Fuzzy logic expert system for evaluating the activity of university teachers,. Int J Assess Tools Educ 8(4):991–1008. https://doi.org/10.21449/ijate.1025690
Correa-Caicedo PJ et al (2021) GPS data correction based on fuzzy logic for tracking land vehicles. Mathematics 9(21). https://doi.org/10.3390/math9212818
Ivanova M, Petkova P, Petkov N (2021) Machine learning and fuzzy logic in electronics: applying intelligence in practice. Electron 10(22):1–29. https://doi.org/10.3390/electronics10222878
Kumar A, Sharma A, Nayyar A (2020) Fuzzy logic based hybrid model for automatic extractive text summarization. In: ACM international conference proceeding series, pp 7–15. https://doi.org/10.3390/electronics10222878
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Al-Jaf, K., Mahmud, H., Öz, C. (2024). Fuzzy Logic-Based Metric for Machine Translation Evaluation. In: Yang, XS., Sherratt, S., Dey, N., Joshi, A. (eds) Proceedings of Ninth International Congress on Information and Communication Technology. ICICT 2024 2024. Lecture Notes in Networks and Systems, vol 1013. Springer, Singapore. https://doi.org/10.1007/978-981-97-3559-4_18
Download citation
DOI: https://doi.org/10.1007/978-981-97-3559-4_18
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-3558-7
Online ISBN: 978-981-97-3559-4
eBook Packages: EngineeringEngineering (R0)