Abstract
Adverse drug reaction (ADR) detection is an essential task in the medical field, as ADRs have a gravely detrimental impact on patients’ health and the healthcare system. Due to a large number of people sharing information on social media platforms, an increasing number of efforts focus on social media data to carry out effective ADR detection. Despite having achieved impressive performance, the existing methods of ADR detection still suffer from three main challenges. Firstly, researchers have consistently ignored the interaction between domain keywords and other words in the sentence. Secondly, social media datasets suffer from the challenges of low annotated data. Thirdly, the issue of sample imbalance is commonly observed in social media datasets. To solve these challenges, we propose the Knowledge Enhanced Shallow and Deep Transformer (KESDT) model for ADR detection. Specifically, to cope with the first issue, we incorporate the domain keywords into the Transformer model through a shallow fusion manner, which enables the model to fully exploit the interactive relationships between domain keywords and other words in the sentence. To overcome the low annotated data, we integrate the synonym sets into the Transformer model through a deep fusion manner, which expands the size of the samples. To mitigate the impact of sample imbalance, we replace the standard cross entropy loss function with the focal loss function for effective model training. We conduct extensive experiments on three public datasets including TwiMed, Twitter, and CADEC. The proposed KESDT outperforms state-of-the-art baselines on F1 values, with relative improvements of 4.87%, 47.83%, and 5.73% respectively, which demonstrates the effectiveness of our proposed KESDT.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Baber, N.: International conference on harmonisation of technical requirements for registration of pharmaceuticals for human use (ICH). Br. J. Clin. Pharmacol. 37(5), 401 (1994)
Kanchan, S., Gaidhane, A.: Social media role and its impact on public health: a narrative review. Cureus 15(1) (2023)
Zhang, T., Lin, H., Xu, B., Yang, L., Wang, J., Duan, X.: Adversarial neural network with sentiment-aware attention for detecting adverse drug reactions. J. Biomed. Inform. 123, 103896 (2021)
Sarker, A., Gonzalez, G.: Portable automatic text classification for adverse drug reaction detection via multi-corpus training. J. Biomed. Inform. 53, 196–207 (2015)
Yadav, S., Ekbal, A., Saha, S., Bhattacharyya, P.: A unified multi-task adversarial learning framework for pharmacovigilance mining. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 5234–5245 (2019)
Chowdhury, S., Zhang, C., Yu, P.S.: Multi-task pharmacovigilance mining from social media posts. In: Proceedings of the 2018 World Wide Web Conference, pp. 117–126 (2018)
Huang, J.Y., Lee, W.P., Lee, K.D.: Predicting adverse drug reactions from social media posts: data balance, feature selection and deep learning. In: Healthcare, vol. 10, p. 618. MDPI (2022)
Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017)
Aljohani, N.R., Fayoumi, A., Hassan, S.U.: A novel focal-loss and class-weight-aware convolutional neural network for the classification of in-text citations. J. Inf. Sci. 49(1), 79–92 (2023)
Kuhn, M., Campillos, M., Letunic, I., Jensen, L.J., Bork, P.: A side effect resource to capture phenotypic effects of drugs. Mol. Syst. Biol. 6(1), 343 (2010)
Benton, A., et al.: Identifying potential adverse effects using the web: a new approach to medical hypothesis generation. J. Biomed. Inform. 44(6), 989–996 (2011)
Yates, A., Goharian, N.: ADRTrace: detecting expected and unexpected adverse drug reactions from user reviews on social media sites. In: Serdyukov, P., et al. (eds.) ECIR 2013. LNCS, vol. 7814, pp. 816–819. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-36973-5_92
Bian, J., Topaloglu, U., Yu, F.: Towards large-scale twitter mining for drug-related adverse events. In: Proceedings of the 2012 International Workshop on Smart Health and Wellbeing, pp. 25–32 (2012)
Patki, A., et al.: Mining adverse drug reaction signals from social media: going beyond extraction. Proc. BioLinkSig 2014, 1–8 (2014)
Rastegar-Mojarad, M., Elayavilli, R.K., Yu, Y., Liu, H.: Detecting signals in noisy data-can ensemble classifiers help identify adverse drug reaction in tweets. In: Proceedings of the Social Media Mining Shared Task Workshop at the Pacific Symposium on Biocomputing (2016)
Zhang, X., Lin, H., Yang, L., Xu, B., Diao, Y., Ren, L.: Dual part-pooling attentive networks for session-based recommendation. Neurocomputing 440, 89–100 (2021)
Zhang, X., et al.: Price does matter! modeling price and interest preferences in session-based recommendation. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1684–1693 (2022)
Zhang, X., et al.: Dynamic intent-aware iterative denoising network for session-based recommendation. Inf. Process. Manag. 59(3), 102936 (2022)
Huynh, T., He, Y., Willis, A., Rüger, S.: Adverse drug reaction classification with deep neural networks. Coling (2016)
Alimova, I., Solovyev, V.: Interactive attention network for adverse drug reaction classification. In: Ustalov, D., Filchenkov, A., Pivovarova, L., Žižka, J. (eds.) AINL 2018. CCIS, vol. 930, pp. 185–196. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01204-5_18
Wu, C., Wu, F., Liu, J., Wu, S., Huang, Y., Xie, X.: Detecting tweets mentioning drug name and adverse drug reaction with hierarchical tweet representation and multi-head self-attention. In: Proceedings of the 2018 EMNLP Workshop SMM4H: the 3rd Social Media Mining for Health Applications Workshop and Shared Task, pp. 34–37 (2018)
Raval, S., Sedghamiz, H., Santus, E., Alhanai, T., Ghassemi, M., Chersoni, E.: Exploring a unified sequence-to-sequence transformer for medical product safety monitoring in social media. In: Findings of the Association for Computational Linguistics: EMNLP 2021, pp. 3534–3546 (2021)
Li, Z., Yang, Z., Luo, L., Xiang, Y., Lin, H.: Exploiting adversarial transfer learning for adverse drug reaction detection from texts. J. Biomed. Inform. 106, 103431 (2020)
Wu, L., et al.: Graph neural networks for natural language processing: a survey. Found. Trends® Mach. Learn. 16(2), 119–328 (2023)
Kwak, H., Lee, M., Yoon, S., Chang, J., Park, S., Jung, K.: Drug-disease graph: predicting adverse drug reaction signals via graph neural network with clinical data. In: Lauw, H.W., Wong, R.C.-W., Ntoulas, A., Lim, E.-P., Ng, S.-K., Pan, S.J. (eds.) PAKDD 2020. LNCS (LNAI), vol. 12085, pp. 633–644. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-47436-2_48
Shen, C., Li, Z., Chu, Y., Zhao, Z.: Gar: graph adversarial representation for adverse drug event detection on twitter. Appl. Soft Comput. 106, 107324 (2021)
Gao, Y., Ji, S., Zhang, T., Tiwari, P., Marttinen, P.: Contextualized graph embeddings for adverse drug event detection. In: Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2022, Grenoble, 19–23 September 2022, Proceedings, Part II, pp. 605–620. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-26390-3_35
Mozzicato, P.: Meddra: an overview of the medical dictionary for regulatory activities. Pharmaceut. Med. 23, 65–75 (2009)
Liu, W., Fu, X., Zhang, Y., Xiao, W.: Lexicon enhanced chinese sequence labeling using bert adapter. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 5847–5858 (2021)
Alvaro, N., et al.: Twimed: twitter and pubmed comparable corpus of drugs, diseases, symptoms, and their relations. JMIR Publ. Health Surveill. 3(2), e6396 (2017)
Karimi, S., Metke-Jimenez, A., Kemp, M., Wang, C.: Cadec: a corpus of adverse drug event annotations. J. Biomed. Inform. 55, 73–81 (2015)
Sarker, A., Nikfarjam, A., Gonzalez, G.: Social media mining shared task workshop. In: Biocomputing 2016: Proceedings of the Pacific Symposium, pp. 581–592. World Scientific (2016)
Acknowledgement
This work is partially supported by grant from the Natural Science Foundation of China (No. 62076046, No.62006130), Inner Monoglia Science Foundation (No.2022MS06028). This work is also supported by the National and Local Joint Engineering Research Center of Intelligent Information Processing Technology for Mongolian and the Inner Mongolia Directly College and University Scientific Basic in 2022.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Qiu, Y., Zhang, X., Wang, W., Zhang, T., Xu, B., Lin, H. (2023). KESDT: Knowledge Enhanced Shallow and Deep Transformer for Detecting Adverse Drug Reactions. In: Liu, F., Duan, N., Xu, Q., Hong, Y. (eds) Natural Language Processing and Chinese Computing. NLPCC 2023. Lecture Notes in Computer Science(), vol 14303. Springer, Cham. https://doi.org/10.1007/978-3-031-44696-2_47
Download citation
DOI: https://doi.org/10.1007/978-3-031-44696-2_47
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44695-5
Online ISBN: 978-3-031-44696-2
eBook Packages: Computer ScienceComputer Science (R0)