Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article
Free access
Just Accepted

Methodical Systematic Review of Abstractive Summarization and Natural Language Processing Models for Biomedical Health Informatics: Approaches, Metrics and Challenges

Online AM: 31 May 2023 Publication History

Abstract

Text summarization tasks are primarily very useful for decision support systems and provide a source for useful data for training of bots as they can reduce and retain the useful information from the large corpus. This review article is for studying the literature that already exists in context of abstractive summarization and application of NLP language models in biomedical and associated healthcare applications. In past decade with trends like bigdata, IOT, enormous amount of data is getting processed in all structured, unstructured and semi structured formats. This review provides a comprehensive literature survey in research trends for abstractive summarization, foundations of machine translation and evolution of language models. This review identifies the potential of language model to provide a possible methodology for improving the performance and accuracy of various tasks in summarization. Deep neural network-based language models have now been the widely accepted state of art for various abstractive summarization and there exists an enormous scope to improvise and tune the language models for domain specific use case. This study shows current systems lack in faithfulness to original content and control of degree of hallucination. This review also details on the evaluation criteria and need for automated metrics and attempts to provide guideline for evaluation for abstractive summarization for health informatics.

References

[1]
Alqudsi, A., Omar, N., & Shaker, K. (2014). Arabic machine translation: a survey. Artificial Intelligence Review, 42(4), 549–572. https://doi.org/10.1007/s10462-012-9351-1
[2]
Alshaina, S., John, A., & Nath, A. G. (2017). Multi-document abstractive summarization based on predicate argument structure. 2017 IEEE International Conference on Signal Processing, Informatics, Communication and Energy Systems, SPICES 2017. https://doi.org/10.1109/SPICES.2017.8091339
[3]
Bagheri, A., Sammani, A., van der Heijden, P.G.M. et al. ETM: Enrichment by topic modeling for automated clinical sentence classification to detect patients’ disease history. J Intell Inf Syst 55, 329–349 (2020). https://doi.org/10.1007/s10844-020-00605-w
[4]
Banerjee, S., Mitra, P., & Sugiyama, K. (2015). Multi-document abstractive summarization using ILP based multi-sentence compression. IJCAI International Joint Conference on Artificial Intelligence, 2015-Janua(Ijcai), 1208–1214.
[5]
Barzilay, R., & Lee, L. (2003). Learning to paraphrase. June, 16–23. https://doi.org/10.3115/1073445.1073448
[6]
Chu, C., & Dabre, R. (2019). Multilingual Multi-Domain Adaptation Approaches for Neural Machine Translation. 1–9. http://arxiv.org/abs/1906.07978
[7]
Chu, C., Dabre, R., & Kurohashi, S. (2018). A comprehensive empirical comparison of domain adaptation methods for neural machine translation. Journal of Information Processing, 26, 529–538. https://doi.org/10.2197/ipsjjip.26.529
[8]
Clark, C., Lee, K., & Zettlemoyer, L. (n.d.). Deep contextualized word representations.
[9]
Conneau, A., Kruszewski, G., Lample, G., Barrault, L., & Baroni, M. (2018). What you can cram into a single ∃&!#* vector: Probing sentence embeddings for linguistic properties. ACL 2018 - 56th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers), 1, 2126–2136. https://doi.org/10.18653/v1/p18-1198
[10]
Dale, R., Moisl, H., & Somers, H. (2001). Handbook of Natural Language Processing. Computational Linguistics, 27(4), 602–603. https://doi.org/10.1162/coli.2000.27.4.602
[11]
Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. NAACL HLT 2019 - 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference, 1(Mlm), 4171–4186.
[12]
Dong, L., Yang, N., Wang, W., Wei, F., Liu, X., Wang, Y., Gao, J., Zhou, M., & Hon, H.-W. (2019). Unified Language Model Pre-training for Natural Language Understanding and Generation. NeurIPS. http://arxiv.org/abs/1905.03197
[13]
Dong, Y. (2018). A Survey on Neural Network-Based Summarization Methods. 1–16. http://arxiv.org/abs/1804.04589
[14]
Dorr, B. J., Jordan, P. W., & Benoit, J. W. (1999). A Survey of Current Paradigms in Machine Translation 1 Introduction 2 The History of MT. In Advances in computers (Vol. 49).
[15]
Elsherif, H. M., & Soomro, T. R. (2017). Perspectives of arabic machine translation. Journal of Engineering Science and Technology, 12(9), 2315–2332.
[16]
Farajian, M. A., Turchi, M., Negri, M., & Federico, M. (2018). Multi-Domain Neural Machine Translation through Unsupervised Adaptation. 1(1), 127–137. https://doi.org/10.18653/v1/w17-4713
[17]
Ferreira, R., De Souza Cabral, L., Lins, R. D., Pereira E Silva, G., Freitas, F., Cavalcanti, G. D. C., Lima, R., Simske, S. J., & Favaro, L. (2013). Assessing sentence scoring techniques for extractive text summarization. Expert Systems with Applications, 40(14), 5755–5764. https://doi.org/10.1016/j.eswa.2013.04.023
[18]
Ganesan, K., Zhai, C. X., & Han, J. (2010). Opinosis: A graph-based approach to abstractive summarization of highly redundant opinions. Coling 2010 - 23rd International Conference on Computational Linguistics, Proceedings of the Conference, 2, 340–348.
[19]
Gehrmann, S., Deng, Y., & Rush, A. M. (2020). Bottom-up abstractive summarization. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018, 4098–4109. https://doi.org/10.18653/v1/d18-1443
[20]
Genest, P.-E. (2011). Framework for abstractive summarization using text-to-text generation. Workshop on Monolingual Text-To-Text Generation, June, 64–73. http://dl.acm.org/citation.cfm?id=2107687
[21]
Gupta, S., & Gupta, S. K. (2019). Abstractive summarization: An overview of the state of the art. Expert Systems with Applications, 121(2018), 49–65. https://doi.org/10.1016/j.eswa.2018.12.011
[22]
Habernal, I., Wachsmuth, H., Gurevych, I., & Stein, B. (2018). The argument reasoning comprehension task: Identification and reconstruction of implicitwarrants. NAACL HLT 2018 - 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference, 1, 1930–1940. https://doi.org/10.18653/v1/n18-1175
[23]
Hameed, A. Z. (2023). A hybrid Fifth Generation based approaches on extracting and analyzing customer requirement through online mode in healthcare industry. Computers and Electrical Engineering, 106, 108550.
[24]
Harabagiu, S. M., & Finley, L. (2001). Generating Single and Multi-Document Summaries with GIST EXTER.
[25]
Howard, J., & Ruder, S. (2018). Universal language model fine-tuning for text classification. ACL 2018 - 56th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers), 1, 328–339. https://doi.org/10.18653/v1/p18-1031
[26]
Hutchins, J. (1986). Strategies and methods since the mid 1960s. Machine Translation: Past, Present, Future.
[27]
Hutchins, J. (2003). Example based machine translation – a review and commentary.
[28]
Hutchins, J. (2012). Machine Translation: General Overview. The Oxford Handbook of Computational Linguistics, 9780199276(May 2018), 1–12. https://doi.org/10.1093/oxfordhb/9780199276349.013.0027
[29]
Hutchins, W. J., & Somers, H. L. (1992). Some other systems and directions of research. In An introduction to machine translation (pp. 313–333).
[30]
Ittycheriah, A. (2010). Statistical machine translation. In Handbook of Natural Language Processing, Second Edition (pp. 409–422). https://doi.org/10.14429/djlit.30.457
[31]
Jhalani, R., & Meena, Y. K. (2017). an Abstractive Approach for Text Summarization. International Journal of Advanced Computational Engineering and Networking, 51, 2320–2106. http://iraj.in
[32]
Jin, H., Wang, T., & Wan, X. (2020). SemSUM: Semantic Dependency Guided Neural Abstractive Summarization. The Thirty-Fourth {AAAI} Conference on Artificial Intelligence, {AAAI} 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, {IAAI} 2020, The Tenth {AAAI} Symposium on Educational Advances in Artificial Intelligence, {EAAI, 8026–8033. https://aaai.org/ojs/index.php/AAAI/article/view/6312
[33]
Kingma, D. P., & Ba, J. L. (2015). Adam: A method for stochastic optimization. 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, 1–15.
[34]
Kou, X., Yang, Y., Wang, Y., Zhang, C., Chen, Y., Tong, Y., Zhang, Y., & Bai, J. (2020). Improving BERT with Self-Supervised Attention. http://arxiv.org/abs/2004.03808
[35]
Lee, C. S., Jian, Z. W., & Huang, L. K. (2005). A fuzzy ontology and its application to news summarization. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 35(5), 859–880. https://doi.org/10.1109/TSMCB.2005.845032
[36]
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2019). BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. http://arxiv.org/abs/1910.13461
[37]
Li, Chen, Liu, F., Weng, F., & Liu, Y. (2013). Document summarization via guided sentence compression. EMNLP 2013 - 2013 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference, October, 490–500.
[38]
Li, Chenliang, Xu, W., Li, S., & Gao, S. (2018). Guiding generation for abstractive text summarization based on key information guide network. NAACL HLT 2018 - 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference, 2(2016), 55–60. https://doi.org/10.18653/v1/n18-2009
[39]
Li, X., Wang, B., Wang, Y. et al. Weakly supervised prototype topic model with discriminative seed words: modifying the category prior by self-exploring supervised signals. Soft Comput 27, 5397–5410 (2023). https://doi.org/10.1007/s00500-022-07771-9
[40]
Liu, F., Flanigan, J., Thomson, S., Sadeh, N., & Smith, N. A. (2015). Toward abstractive summarization using semantic representations. NAACL HLT 2015 - 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference, 1077–1086. https://doi.org/10.3115/v1/n15-1114
[41]
Liu, Yang, & Lapata, M. (2020). Text summarization with pretrained encoders. EMNLP-IJCNLP 2019 - 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, Proceedings of the Conference, 3730–3740. https://doi.org/10.18653/v1/d19-1387
[42]
Liu, Yinhan, Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. 1. http://arxiv.org/abs/1907.11692
[43]
Liu, J., Capurro, D., Nguyen, A., & Verspoor, K. (2022). “Note Bloat” impacts deep learning-based NLP models for clinical prediction tasks. Journal of biomedical informatics, 133, 104149.
[44]
Li, K., Zhou, C., Luo, X. R., Benitez, J., & Liao, Q. (2022). Impact of information timeliness and richness on public engagement on social media during COVID-19 pandemic: An empirical investigation based on NLP and machine learning. Decision Support Systems, 162, 113752.
[45]
Manning, C. D., & Hinrich, S. (1999). Foundations of Statistical Natural Language Processing. https://www.cs.vassar.edu/∼cs366/docs/Manning_Schuetze_StatisticalNLP.pdf
[46]
Maynez, J., Narayan, S., Bohnet, B., & McDonald, R. (2020). On Faithfulness and Factuality in Abstractive Summarization. i, 1906–1919. http://arxiv.org/abs/2005.00661
[47]
McCann, B., Bradbury, J., Xiong, C., & Socher, R. (2017). Learned in translation: Contextualized word vectors. Advances in Neural Information Processing Systems, 2017-Decem(Nips), 6295–6306.
[48]
Merity, S., Keskar, N. S., & Socher, R. (2018). Regularizing and optimizing LSTM language models. 6th International Conference on Learning Representations, ICLR 2018 - Conference Track Proceedings.
[49]
Moawad, I. F., & Aref, M. (2012). Semantic graph reduction approach for abstractive Text Summarization. Proceedings - ICCES 2012: 2012 International Conference on Computer Engineering and Systems, May 2014, 132–138. https://doi.org/10.1109/ICCES.2012.6408498
[50]
Nagao, M. (1984). A FRAMEWORK OF A MECHANICAL TRANSLATION BETWEEN JAPANESE AND ENGLISH BY ANALOGY PRINCIPLE.
[51]
Och, F. J. (2003). A Systematic Comparison of Various. Computational Linguistics.
[52]
Peters, M. E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018). Improving Language Understanding by. OpenAI, 1–10. https://gluebenchmark.com/leaderboard%0Ahttps://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf
[53]
Reimers, N., & Gurevych, I. (2020). Sentence-BERT: Sentence embeddings using siamese BERT-networks. EMNLP-IJCNLP 2019 - 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, Proceedings of the Conference, 3982–3992. https://doi.org/10.18653/v1/d19-1410
[54]
Ribeiro, L. F. R., Gardent, C., & Gurevych, I. (2020). Enhancing AMR-to-text generation with dual graph representations. EMNLP-IJCNLP 2019 - 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, Proceedings of the Conference, 3183–3194. https://doi.org/10.18653/v1/d19-1314
[55]
Ribelles, N., Jerez, J. M., Rodriguez-Brazzarola, P., Jimenez, B., Diaz-Redondo, T., Mesa, H., ... & Alba, E. (2021). Machine learning and natural language processing (NLP) approach to predict early progression to first-line treatment in real-world hormone receptor-positive (HR+)/HER2-negative advanced breast cancer patients. European Journal of Cancer, 144, 224-231.
[56]
Ruder, S. (2019). Neural Transfer Learning for Natural Language Processing.
[57]
Rush, A. M., Chopra, S., & Weston, J. (2015). A neural attention model for sentence summarization. Conference Proceedings - EMNLP 2015: Conference on Empirical Methods in Natural Language Processing, 379–389.
[58]
SAWAF, H., SHIHADAH, M., & YAGHI, M. (2009). Hybrid machine translation. https://patentscope.wipo.int/search/en/detail.jsf?docId=WO2010046782&tab=PCTBIBLIO
[59]
See, A., Liu, P. J., & Manning, C. D. (2017). Get to the point: Summarization with pointer-generator networks. ACL 2017 - 55th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers), 1, 1073–1083. https://doi.org/10.18653/v1/P17-1099
[60]
Seo, P. H., Lin, Z., Cohen, S., Shen, X., & Han, B. (2016). Hierarchical Attention Networks. ArXiv, 1480–1489. http://arxiv.org/abs/1606.02393
[61]
Shen, D., Chen, Z., Yang, Q., Zeng, H. J., Zhang, B., Lu, Y., & Ma, W. Y. (2004). Web-page classification through summarization. Proceedings of Sheffield SIGIR - Twenty-Seventh Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 242–249. https://doi.org/10.1145/1008992.1009035
[62]
Sloculn, J. (1985). a Survey of Machine Translation : Its History, Current Status, and Future Prospects. Computational Linguistics, 11(1), 1–17.
[63]
Song, K., Tan, X., Qin, T., Lu, J., & Liu, T. Y. (2019). MASS: Masked sequence to sequence pre-training for language generation. 36th International Conference on Machine Learning, ICML 2019, 2019-June, 10384–10394.
[64]
Sunitha, C., Jaya, A., & Ganesh, A. (2016). A Study on Abstractive Summarization Techniques in Indian Languages. Procedia Computer Science, 87, 25–31. https://doi.org/10.1016/j.procs.2016.05.121
[65]
Uria, B., Cote, M. A., Gregor, K., Murray, I., & Larochelle, H. (2016). Neural autoregressive distribution estimation. Journal of Machine Learning Research, 17, 1–37.
[66]
Wang, A., Michael, J., Hill, F., Levy, O., Singh, A., & Bowman, S. R. (2019). SuperGLUE : A Stickier Benchmark for General-Purpose Language Understanding Systems. 2019(July), 1–29.
[67]
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. R. (2019a). GLUE : A M ULTI -T ASK B ENCHMARK AND A NALYSIS P LATFORM FOR N ATURAL L ANGUAGE U NDERSTAND -. Proceeding, 1–20. https://doi.org/10.18653/v1/W18-5446
[68]
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. R. (2019b). Glue: A multi-task benchmark and analysis platform for natural language understanding. 7th International Conference on Learning Representations, ICLR 2019, 353–355.
[69]
Weng, L. (2019). Generalized Language Models. Lilianweng.Github.Io/Lil-Log. https://lilianweng.github.io/lil-log/2019/01/31/generalized-language-models.html
[70]
Wołk, K., & Marasek, K. (2015). Neural-based Machine Translation for Medical Text Domain. Based on European Medicines Agency Leaflet Texts. Procedia Computer Science, 64(March 2016), 2–9. https://doi.org/10.1016/j.procs.2015.08.456
[71]
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019). XLNet: Generalized Autoregressive Pretraining for Language Understanding. NeurIPS, 1–18. http://arxiv.org/abs/1906.08237
[72]
Zajic, D. M., Dorr, B. J., & Lin, J. (2008). Single-document and multi-document summarization techniques for email threads using sentence compression. 44, 1600–1610. https://doi.org/10.1016/j.ipm.2007.09.007
[73]
Zantout, R., & Guessoum, A. (2000). Arabic Machine Translation: A Strategic Choice for the Arab World. Journal of King Saud University - Computer and Information Sciences, 12(December), 117–144. https://doi.org/10.1016/s1319-1578(00)80005-1
[74]
Zhang, D., Xia, X., Yang, Y., Yang, P., Xie, C., Cui, M., & Liu, Q. (2021). A novel word similarity measure method for IoT-enabled healthcare applications. Future Generation Computer Systems, 114, 209-218.
[75]
Zhu, C., Hinthorn, W., Xu, R., Zeng, Q., Zeng, M., Huang, X., & Jiang, M. (2020). Boosting Factual Correctness of Abstractive Summarization. http://arxiv.org/abs/2003.08612

Cited By

View all
  • (2024)Single-Document Abstractive Text Summarization: A Systematic Literature ReviewACM Computing Surveys10.1145/370063957:3(1-37)Online publication date: 11-Nov-2024
  • (2024)Medical Question Summarization with Entity-driven Contrastive LearningACM Transactions on Asian and Low-Resource Language Information Processing10.1145/365216023:4(1-19)Online publication date: 15-Apr-2024
  • (2024)NLP Based Automated Text Summarization and Translation: A Comprehensive Analysis2024 2nd International Conference on Sustainable Computing and Smart Systems (ICSCSS)10.1109/ICSCSS60660.2024.10624907(528-531)Online publication date: 10-Jul-2024
  • Show More Cited By

Index Terms

  1. Methodical Systematic Review of Abstractive Summarization and Natural Language Processing Models for Biomedical Health Informatics: Approaches, Metrics and Challenges
          Index terms have been assigned to the content through auto-classification.

          Recommendations

          Comments

          Please enable JavaScript to view thecomments powered by Disqus.

          Information & Contributors

          Information

          Published In

          cover image ACM Transactions on Asian and Low-Resource Language Information Processing
          ACM Transactions on Asian and Low-Resource Language Information Processing Just Accepted
          EISSN:2375-4702
          Table of Contents
          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          Online AM: 31 May 2023
          Accepted: 22 May 2023
          Revised: 05 May 2023
          Received: 18 March 2023

          Check for updates

          Author Tags

          1. NLP
          2. abstractive summarization
          3. sentence compression
          4. sentence fusion
          5. document summarization
          6. language model
          7. ROUGE

          Qualifiers

          • Research-article

          Contributors

          Other Metrics

          Bibliometrics & Citations

          Bibliometrics

          Article Metrics

          • Downloads (Last 12 months)305
          • Downloads (Last 6 weeks)37
          Reflects downloads up to 24 Nov 2024

          Other Metrics

          Citations

          Cited By

          View all
          • (2024)Single-Document Abstractive Text Summarization: A Systematic Literature ReviewACM Computing Surveys10.1145/370063957:3(1-37)Online publication date: 11-Nov-2024
          • (2024)Medical Question Summarization with Entity-driven Contrastive LearningACM Transactions on Asian and Low-Resource Language Information Processing10.1145/365216023:4(1-19)Online publication date: 15-Apr-2024
          • (2024)NLP Based Automated Text Summarization and Translation: A Comprehensive Analysis2024 2nd International Conference on Sustainable Computing and Smart Systems (ICSCSS)10.1109/ICSCSS60660.2024.10624907(528-531)Online publication date: 10-Jul-2024
          • (2024)Airline reviews processing: Abstractive summarization and rating-based sentiment classification using deep transfer learningInternational Journal of Information Management Data Insights10.1016/j.jjimei.2024.1002384:2(100238)Online publication date: Nov-2024
          • (2024)From rule-based models to deep learning transformers architectures for natural language processing and sign language translation systems: survey, taxonomy and performance evaluationArtificial Intelligence Review10.1007/s10462-024-10895-z57:10Online publication date: 29-Aug-2024

          View Options

          View options

          PDF

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          Login options

          Full Access

          Media

          Figures

          Other

          Tables

          Share

          Share

          Share this Publication link

          Share on social media