Abstract
Named entity recognition aims to identify and mark entities with specific meanings in text. It is a key technology to further extract entity relationships and mine other potential information in natural language processing. At present, the methods based on machine learning and deep learning have been widely used in the research of named entity recognition, but most learning models use feature extraction based on word and character level. The word preprocessing of this kind of model often ignores the context semantic information of the target word and can not realize polysemy. In addition, the loss of semantic information and limited training data greatly limit the improvement of model performance and generalization ability. In order to solve the above problems and improve the efficiency of named entity recognition technology in Chinese text, this paper constructs a multi-task BERT-BiLSTM-AM-CRF intelligent processing model, uses Bert to extract the dynamic word vector combined with context information, and inputs the results into CRF layer for decoding after further training through BiLSTM module. After attention mechanism network, the model can learn together on two Chinese datasets, Finally, CRF classifies and extracts the observation annotation sequence to get the final result. Compared with many previous single task models, the F1 score of this multi-task model in MASR and people’s daily datasets has been significantly improved (0.55% and 3.41%), which demonstrates the effectiveness of multi-task learning for Chinese named entity recognition.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Xie X, Fu Y, Jin H, Zhao Y, Cao W (2020) A novel text mining approach for scholar information extraction from web content in chinese. Futur Gener Comput Syst 111:859–872
Xiang L, Sun X, Luo G, Xia B (2014) Linguistic steganalysis using the features derived from synonym frequency. Multimed Tools Appl 71(3):1893–1911
Sun C, Yang Z, Wang L, Zhang Y, Lin H, Wang J (2021) Biomedical named entity recognition using bert in the machine reading comprehension framework. J. Biomedical Informatics 118:103799
Zhai Z, Nguyen DQ, Akhondi S, Thorne C, Verspoor K (2019) Improving Chemical Named Entity Recognition in Patents with Contextualized Word Embeddings. Proceedings of the 18th BioNLP Workshop and Shared Task
Ivan L, Nicolas P, Xavier T (2020) Terminologies augmented recurrent neural network model for clinical named entity recognition. J. Biomedical Informatics 102:103356
Zhou S, Tan B (2020) Electrocardiogram soft computing using hybrid deep learning cnn-elm. Appl Soft Comput 86:105778
He S, Li Z, Tang Y, Liao Z, Li F, Lim S (2020) Parameters compressing in deep learning. Cmc-computers Materials & Continua 62(1):321–336
Habibi M, Weber L, Neves M, Wiegandt DL, Leser U (2017) Deep learning with word embeddings improves biomedical named entity recognition. Bioinformatics 33(14):37–48
Lample G, Ballesteros M, Subraman S (2016) Neural architectures for namedentity recognition. Proceedings of NAACL-HLT, 260–270
Ma X, Hovy EK (2016) End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics
Chen Y, Zhou C, Li T, Wu H, Zhao X, Ye K, Liao J (2019) Named entity recognition from chinese adverse drug event reports with lexical feature based bilstm-crf and tri-training. J. Biomedical Informatics 96:103252
Li Z, Li Q, Zou X, Ren J (2021) Causality extraction based on self-attentive bilstm-crf with transferred embeddings. Neurocomputing 423:207–219
Putra FM, Retno K, Adi W (2021) Sentiment analysis using word2vec and long short-term memory (lstm) for indonesian hotel reviews. Procedia Comp Sci 179:728–735
Quang-Thai H, Trinh-Trung-Duong N, Nguyen QKL, Yu-Yen O (2021) Fad-bert: Improved prediction of fad binding sites using pre-training of deep bidirectional transformers. Comput Biol Med 131:104258
Tang X, Cao W, Tang H, Deng T, Mei J, Liu Y, Shi C, Xia M, Zeng Z (2022) Cost-efficient workflow scheduling algorithm for applications with deadline constraint on heterogeneous clouds. IEEE Trans. Parallel Distrib Syst 33(9):2079–2092
Tang X, Shi C, Deng T, Wu Z, Yang L (2021) Parallel random matrix particle swarm optimization scheduling algorithms with budget constraints on cloud computing systems. Appl Soft Comput 113:107914
Fukuda K, Tsunoda T, Tamura A et al (1998) Toward information extraction: identifying protein names from biological papers. Pac Symp Biocomput 707(18):707–718
Hanisch D, Fundel K, Mevissen HT et al (2005) Prominer: rule-based protein and gene entity recognition. BMC bioinformatics 6(1):14
Lee KJ, Hwang YS, Kim S et al (2004) Biomedical named entity recognition using two-phase model based on svms. J. Biomedical Informatics 37(6):436–447
Satyajit N, Justin D (2022) Factored latent-dynamic conditional random fields for single and multi-label sequence modeling. Pattern Recogn 122:108236
Chen H, Sun F, Yuan J, Huan Y (2021) Mirrored conditional random field model for object recognition in indoor environments. Inf Sci 551:291–303
Liu X, Zhou Y, Wang Z (2021) Deep neural network-based recognition of entities in chinese online medical inquiry texts. Futur Gener Comput Syst 114:581–604
De Oliveira DM, Laender AHF, Veloso A et al (2013) FS-NER: A lightweight filter-stream approach to named entity recognition on twitter data. Proceedings of the 22nd International Conference on World Wide Web
Liu P, Guo Y, Wang F, Li G (2022) Chinese named entity recognition: The state of the art. Neurocomputing 473:37–53
Jia Y, Xu X (2018) Chinese named entity recognition based on CNN-BiLSTM-CRF. IEEE 9th International Conference on Software Engineering and Service Science (ICSESS)
Zhao S, Cai Z, Chen H, Wang Y, Liu F, Liu A (2019) Adversarial training based lattice lstm for chinese clinical named entity recognition. J. Biomedical Informatics 99:103290
Chang N, Zhong J, Li Q, Zhu J (2020) A mixed semantic features model for chinese ner with characters and words. In: ECIR 2020: Advances in Information Retrieval, pp. 356–368. Springer, Heidelberg
Dai Z, Wang X, Ni P, Li Y, Li G, Bai X (2019) Named entity recognition using BERT BiLSTM CRF for Chinese electronic health records. The 12th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI)
Murugesan G, Abdulkadhar S, Bhasuran B, Natarajan J (2017) Bcc-ner: bidirectional, contextual clues named entity tagger for gene/protein mention recognition. J. Bioinform Sys. Biology 7:1–8
Cheng P, Dai J, Liu J (2022) Catvrnn: Generating category texts via multi-task learning. Knowl-Based Syst 244:108491
Xu K, Zhou Z, Gong T, Hao T, Liu W (2018) Sblc: a hybrid model for disease named entity recognition based on semantic bidirectional lstms and conditional random fields. BMC Med Inform Decis Mak 18:114
Graves A, Mohamed AR, Hinton G (2013) Speech recognition with deep recurrent neural networks. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing
Xu L, Dong Q, Yu C, Tian Y, Liu W, Li L, Zhang X (2020) Cluener2020: Fine-grained name entity recognition for chinese. arXiv preprint arXiv:2001.04351
Acknowledgements
This research was partially funded by National Natural Science Foundation of China (Grant No. 61972146), Hunan Provincial Natural Science Foundation of China (Grant No. 2020JJ4376).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Tang, X., Huang, Y., Xia, M. et al. A Multi-Task BERT-BiLSTM-AM-CRF Strategy for Chinese Named Entity Recognition. Neural Process Lett 55, 1209–1229 (2023). https://doi.org/10.1007/s11063-022-10933-3
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-022-10933-3