Abstract
Text similarity detection is one of the significant research problems in the Natural Language Processing field. In this paper, we propose an approach that uses machine learning models with seven character-based similarity measures to classify texts based on similarity. For this purpose, we use character-based similarity measures—Longest Common Substring, Longest Common Subsequence, Ratcliff/Obershelp algorithms, Jaro, Jaro–Winkler, Levenshtein, and Damerau-Levenshtein distances as input of supervised machine learning algorithms. For the similarity detection task, news articles are collected from Azerbaijani news websites and 9600 text pairs are created and manually labeled as similar and non-similar. These text pairs are processed by similarity measures to feed Machine learning algorithms—Support Vector Machine, Random Forest and Multi-layer Perceptron Neural Network. We performed a 10-fold cross-validation process on the dataset and found that the trained Neural Networks model gives the best mean accuracy (96%) in detecting similarity between two text bodies. We demonstrated that our proposed method outperforms results gained from individual character-based similarity measurement.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Dataset and code of the experiments are at https://github.com/EmilKalbaliyev/textsimilarity.
References
Gomaa, W.H., Fahmy, A.A.: A survey of text similarity approaches. Int. J. Comput. Appl. 68, 13–18 (2013). https://doi.org/10.5120/11638-7118
Chandrasekaran, D., Mago, V.: Evolution of semantic similarity—a survey. J. Assoc. Comput. Mach. 37(4), 1–29 (2020), Article 111. https://doi.org/10.1145/1122445.1122456
Mihalcea, R., Corley, C., Strapparava, C.: Corpus based and knowledge-based measures of text semantic similarity. In: Proceedings of the 21st National Conference on Artificial intelligence, vol. 1, pp. 775–780 (2006). https://doi.org/10.5555/1597538.1597662
Wang, Z., Mi, H., Ittycheriah, A.: Sentence similarity learning by lexical decomposition and composition. In: Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers. The COLING 2016 Organizing Committee, pp. 1340–1349 (2016)
Shao, Y.: HCTI at SemEval-2017 Task 1: use convolutional neural network to evaluate Semantic Textual Similarity. In: Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017) (2017). https://doi.org/10.18653/v1/s17-2016
He, H., Gimpel, K., Lin, J.: Multi-perspective sentence similarity modeling with convolutional neural networks. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (2015). https://doi.org/10.18653/v1/d15-1181
Tien, N., Le, N., Tomohiro, Y., Tatsuya, I.: Sentence modeling via multiple word embeddings and multi-level comparison for semantic textual similarity. Inf. Process. Manage. 56, (2019). https://doi.org/10.1016/j.ipm.2019.102090
Islam, A., Inkpen, D.: Semantic text similarity using corpus-based word similarity and string similarity. ACM Trans. Knowl. Discov. Data 2, 1–25 (2008). https://doi.org/10.1145/1376815.1376819
Hassan, B., Abdelrahman, S., Bahgat, R., Farag, I.: UESTS: an unsupervised ensemble semantic textual similarity method. IEEE Access 7, 85462–85482 (2019). https://doi.org/10.1109/ACCESS.2019.2925006
Ruas, T., Grosky, W., Aizawa, A.: Multi-sense embeddings through a word sense disambiguation process. Expert Syst. Appl. 136, 288–303 (2019). https://doi.org/10.1016/j.eswa.2019.06.026
Ratcliff, J.W., Metzener, D.: Pattern matching: the gestalt approach. Dr. Dobb’s J. 46 (1988)
Jaro, M.: Advances in record-linkage methodology as applied to matching the 1985 Census of Tampa, Florida. J. Am. Stat. Assoc. 84, 414–420 (1989). https://doi.org/10.2307/2289924
Winkler, W.E.: String comparator metrics and enhanced decision rules in the Fellegi-Sunter model of record linkage. In: Proceedings of the Section on Survey Research Methods. American Statistical Association, pp. 354–359 (1990)
Levenshtein, V.I.: Binary codes capable of correcting spurious insertions and deletions of ones. Soviet Phys. Doklady 10(8), 707–710 (1966)
Damerau, F.J.: A technique for computer detection and correction of spelling errors. Commun. ACM 7(3), 171–176 (1964). https://doi.org/10.1145/363958.363994
Scikit-learn: Supervised Learning. https://scikit-learn.org/stable/supervised_learning.html
Acknowledgements
This work has been carried out at the Center of Data Analytics and Research at ADA University. We express our deep gratitude to Mardan Safarov, Vasif Vahidov, and Ulvi Mammadli for their assistance in this research work.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Kalbaliyev, E., Rustamov, S. (2021). Text Similarity Detection Using Machine Learning Algorithms with Character-Based Similarity Measures. In: Biele, C., Kacprzyk, J., Owsiński, J.W., Romanowski, A., Sikorski, M. (eds) Digital Interaction and Machine Intelligence. MIDI 2020. Advances in Intelligent Systems and Computing, vol 1376. Springer, Cham. https://doi.org/10.1007/978-3-030-74728-2_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-74728-2_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-74727-5
Online ISBN: 978-3-030-74728-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)