Nothing Special   »   [go: up one dir, main page]
Skip to main content
SpringerLink
Log in

Borrowing wisdom from world: modeling rich external knowledge for Chinese named entity recognition

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Chinese named entity recognition (CNER) is one of the fundamental tasks in natural language processing (NLP), supporting a wide range of downstream NLP tasks for Chinese texts. The recent best-performing CNER works have extensively shown that by using external knowledge, such as lexicons and syntactic dependency features, considerable task improvements can be secured. Nevertheless, we note that current works still fail to sufficiently integrate rich external knowledge to boost the CNER performances further. In this work, we propose to enhance the CNER by incorporating heterogeneous knowledge from the linguistic, syntactic and semantic perspectives. For linguistic source, we consider (1) part-of-speech (POS) tags and (2) multi-granularity lexicons, including characters, words and subwords. For syntactic source, we adopt label-wise character-level syntactic dependency structures. For semantic source, we employ (1) BERT contextualized representations and (2) rich sememe representations from HowNet. We build heterogeneous graphs based on the multi-granularity lexicons and encode them with graph attention neural network (GAT). We also propose an innovative label-aware graph convolutional network (LGCN) for modeling the syntactic dependency arcs and labels simultaneously. Further, we present a sememe composition attention module for better injecting the sememe representations. Our system achieves new state-of-the-art CNER performances over current best baselines on four benchmark datasets. Further in-depth analysis has been conducted to reveal the contribution of each used resource, as well as the strengths of our proposed methods for the task improvements.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

We will make all the data and material publicly available upon the acceptance of our work.

Code availability

We will make all the codes publicly available upon the acceptance of our work.

Notes

  1. https://www.weibo.com/.

  2. https://catalog.ldc.upenn.edu/LDC2011T13.

  3. https://openhownet.thunlp.org/.

  4. https://catalog.ldc.upenn.edu/LDC2016T13.

  5. https://github.com/google-research/bert, base cased version.

References

  1. Bikel DM, Miller S, Schwartz R, Weischedel R (1997) Nymble: a highperformance learning name-finder. In: Proceedings of the conference on applied natural language processing

  2. Bunescu R, Mooney R (2005) A shortest path dependency kernel for relation extraction. In: Proceedings of human language technology conference and conference on empirical methods in natural language processing, pp 724–731

  3. Cao P, Chen Y, Liu K, Zhao J, Liu S (2018) Adversarial transfer learning for Chinese named entity recognition with self-attention mechanism. In: Proceedings of the 2018 conference on empirical methods in natural language processing, pp 182–192

  4. Che W, Wang M, Manning CD, Liu T (2013) Named entity recognition with bilingual constraints. In: Proceedings of the 2013 conference of the North American chapter of the association for computational linguistics: human language technologies, pp 52–62

  5. Chen A, Peng F, Shan R, Sun G (2006) Chinese named entity recognition with conditional probabilistic models. In: Proceedings of the SIGHAN workshop on Chinese language processing, pp 173–176

  6. Chen Y, Xu L, Liu K, Zeng D, Zhao J (2015) Event extraction via dynamic multi-pooling convolutional neural networks. In: Proceedings of the annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing, pp 167–176

  7. Collobert R, Weston J, Bottou L, Karlen M, Kavukcuoglu K, Kuksa PP (2011) Natural language processing (almost) from scratch. J Mach Learn Res 12:2493–2537

    MATH  Google Scholar 

  8. Devlin J, Chang MW, Lee K, Toutanova K (2019) BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics, pp 4171–4186

  9. Diefenbach D, López V, Singh KD, Maret P (2018) Core techniques of question answering systems over knowledge bases: a survey. Knowl Inf Syst 55(3):529–569

    Article  Google Scholar 

  10. Dong C, Zhang J, Zong C, Hattori M, Di H (2016) Character-based LSTM-CRF with radical-level features for Chinese named entity recognition. In: Proceedings of the conference on natural language processing and Chinese computing, pp 239–250

  11. Dozat T, Manning CD (2017) Deep biaffine attention for neural dependency parsing. In: Proceedings of the international conference on learning representations

  12. Fei H, Guo Y, Li B, Ji D, Ren Y (2021) Adversarial shared-private model for cross-domain clinical text entailment recognition. Knowledge-Based Syst 221:106962

    Article  Google Scholar 

  13. Fei H, Ji D, Li B, Liu Y, Ren Y, Li F (2021) Rethinking boundaries: End-to-end recognition of discontinuous mentions with pointer networks. In: Proceedings of the AAAI conference on artificial intelligence, pp 12785–12793

  14. Fei H, Li F, Li B, Ji D (2021) Encoder-decoder based unified semantic role labeling with label-aware syntax. In: Proceedings of the AAAI conference on artificial intelligence, pp 12794–12802

  15. Fei H, Ren Y, Ji D (2020) Improving text understanding via deep syntax-semantics communication. In: Proceedings of the conference on empirical methods in natural language processing, pp 84–93

  16. Fei H, Ren Y, Ji D (2020) Mimic and conquer: Heterogeneous tree structure distillation for syntactic NLP. Findings of the association for computational linguistics: EMNLP 2020:183–193

  17. Fei H, Ren Y, Ji D (2020) Retrofitting structure-aware transformer language model for end tasks. In: Proceedings of the 2020 conference on empirical methods in natural language processing(EMNLP), pp 2151–2161

  18. Fei H, Ren Y, Wu S, Li B, Ji D (2021) Latent target-opinion as prior for document-level sentiment classification: a variational approach from fine-grained perspective. In: Proceedings of the WWW: the web conference, pp 553–564

  19. Fei H, Ren Y, Zhang Y, Ji D, Liang X (2020) Enriching contextualized language model from knowledge graph for biomedical information extraction. Brief Bioinform

  20. Fei H, Wu S, Ren Y, Li F, Ji D (2021) Better combine them together! integrating syntactic constituency and dependency representations for semantic role labeling. In: Findings of the association for computational linguistics: ACL/IJCNLP 2021, pp 549–559

  21. Fei H, Zhang M, Ji D (2020) Cross-lingual semantic role labeling with high-quality translated training corpus. In: Proceedings of the 58th annual meeting of the association for computational linguistics, pp 7014–7026

  22. Fei H, Zhang M, Li B, Ji D (2021) End-to-end semantic role labeling with neural transition-based model. In: Proceedings of the AAAI conference on artificial intelligence, pp 12803–12811

  23. Fei H, Zhang M, Li F, Ji D (2020) Cross-lingual semantic role labeling with model transfer. IEEE/ACM Trans Audio Speech Lang Process 28:2427–2437

    Article  Google Scholar 

  24. Fei H, Zhang Y, Ren Y, Ji D (2020) Latent emotion memory for multi-label emotion classification. In: Proceedings of the AAAI conference on artificial intelligence, pp 7692–7699

  25. Gilmer J, Schoenholz SS, Riley PF, Vinyals O, Dahl GE (2017) Neural message passing for quantum chemistry. In: Proceedings of the international conference on machine learning, pp 1263–1272

  26. Gui T, Zou Y, Zhang Q, Peng M, Fu J, Wei Z, Huang X (2019) A lexicon-based graph neural network for Chinese NER. In: Proceedings of the conference on empirical methods in natural language processing, pp 1040–1050

  27. He H, Sun X (2017) F-score driven max margin neural network for named entity recognition in Chinese social media. In: Proceedings of the 15th conference of the European chapter of the association for computational linguistics, pp 713–718

  28. He J, Wang H (2008) Chinese named entity recognition and word segmentation based on character. In: Proceedings of the SIGHAN Workshop on Chinese language processing

  29. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  30. Huang S, Sun X, Wang H (2017) Addressing domain adaptation for Chinese word segmentation with global recurrent structure. In: Proceedings of the international joint conference on natural language processing, pp 184–193

  31. Isozaki H, Kazawa H (2002) Efficient support vector classifiers for named entity recognition. In: Proceedings of the international conference on computational linguistics

  32. Jie Z, Lu W (2019) Dependency-guided LSTM-CRF for named entity recognition. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing, pp 3862–3872

  33. Jie Z, Muis AO, Lu W (2017) Efficient dependency-guided named entity recognition. In: Singh SP, Markovitch S (eds) Proceedings of the association for the advancement of artificial intelligence, pp 3457–3465

  34. Kim Y (2014) Convolutional neural networks for sentence classification. In: Proceedings of the 2014 conference on empirical methods in natural language processing, pp 1746–1751

  35. Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: Proceedings of the international conference on learning representations

  36. Lafferty JD, McCallum A, Pereira FCN (2001) Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of the international conference on machine learning, pp 282–289

  37. Lample G, Ballesteros M, Subramanian S, Kawakami K, Dyer C (2016) Neural architectures for named entity recognition. In: Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies, pp 260–270

  38. Levow GA (2006) The third international Chinese language processing bakeoff: Word segmentation and named entity recognition. In: Proceedings of the SIGHAN workshop on Chinese language processing, pp 108–117

  39. Li B, Fei H, Ren Y, Ji D (2020) Nominal compound chain extraction: a new task for semantic-enriched lexical chain. In: Proceedings of the natural language processing and Chinese computing, pp 119–131

  40. Li Z, Ding N, Liu Z, Zheng H, Shen Y (2019) Chinese relation extraction with multi-grained information and external linguistic knowledge. In: Proceedings of the annual meeting of the association for computational linguistics, pp 4377–4386

  41. Liu J, Huang M, Zhu X (2010) Recognizing biomedical named entities using skip-chain conditional random fields. In: Proceedings of the workshop on biomedical natural language processing, pp 10–18

  42. Liu L, Shang J, Ren X, Xu FF, Gui H, Peng J, Han J (2018) Empower sequence labeling with task-aware neural language model. In: Proceedings of the association for the advancement of artificial intelligence, pp 5253–5260

  43. Liu Z, Zhu C, Zhao T (2010) Chinese named entity recognition with a sequence labeling approach: Based on characters, or based on words? In: Proceedings of the advanced intelligent computing theories and applications, pp 634–640

  44. Lu Y, Zhang Y, Ji D (2016) Multi-prototype Chinese character embedding. In: Proceedings of the tenth international conference on language resources and evaluation

  45. Ma X, Hovy E (2016) End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. In: Proceedings of the annual meeting of the association for computational linguistics, pp 1064–1074

  46. Marcheggiani D, Titov I (2017) Encoding sentences with graph convolutional networks for semantic role labeling. In: Proceedings of the conference on empirical methods in natural language processing, pp 1506–1515

  47. Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013) Distributed representations of words and phrases and their compositionality. In: Proceedings of the 27th annual conference on neural information orocessing systems, pp 3111–3119

  48. Niu Y, Xie R, Liu Z, Sun M (2017) Improved word representation learning with Sememes. In: Proceedings of the annual meeting of the association for computational linguistics, pp 2049–2058

  49. Passos A, Kumar V, McCallum A (2014) Lexicon infused phrase embeddings for named entity resolution. In: Proceedings of the conference on computational natural language learning, pp 78–86

  50. Peng N, Dredze M (2015) Named entity recognition for Chinese social media with jointly trained embeddings. In: Proceedings of the 2015 conference on empirical methods in natural language processing, pp 548–554

  51. Provilkov I, Emelianenko D, Voita E (2020) BPE-dropout: Simple and effective subword regularization. In: Proceedings of the annual meeting of the association for computational linguistics, pp 1882–1892

  52. Qi F, Huang J, Yang C, Liu Z, Chen X, Liu Q, Sun M (2019) Modeling semantic compositionality with sememe knowledge. In: Proceedings of the annual meeting of the association for computational linguistics, pp 5706–5715

  53. Sasano R, Kurohashi S (2008) Japanese named entity recognition using structural natural language processing. In: Proceedings of the international joint conference on natural language processing

  54. Sui D, Chen Y, Liu K, Zhao J, Liu S (2019) Leverage lexical knowledge for Chinese named entity recognition via collaborative graph network. In: Proceedings of the 2019 conference on empirical methods in natural language processing, pp 3830–3840

  55. Tang Z, Wan B, Yang L (2020) Word-character graph convolution network for Chinese named entity recognition. IEEE/ACM Trans Audio Speech Lang Process 28:1520–1532

    Article  Google Scholar 

  56. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. In: Proceedings of the international conference on neural information processing, pp 5998–6008

  57. Velickovic P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y (2018) Graph attention networks. In: Proceedings of the 6th international conference on learning representations

  58. Wang M, Che W, Manning CD (2013) Effective bilingual constraints for semi-supervised learning of named entity recognizers. In: Proceedings of the association for the advancement of artificial intelligence

  59. Weischedel R, Palmer M, Marcus M, Hovy E, Pradhan S, Ramshaw L, Xue N, Taylor A, Kaufman J, Franchini M, El-Bachouti M, Belvin R, Houston A (2011) Ontonotes release 4.0. LDC2011T03, Philadelphia, Penn.: Linguistic Data Consortium

  60. Wu S, Fei H, Ren Y, Li B, Li F, Ji D (2021) High-order pair-wise aspect and opinion terms extraction with edge-enhanced syntactic graph convolution. IEEE/ACM Trans Audio Speech Lang Process 29:2396–2406

    Article  Google Scholar 

  61. Xue N, Shen L (2003) Chinese word segmentation as LMR tagging. In: Proceedings of the second SIGHAN workshop on Chinese language processing, pp 176–179

  62. Yang J, Teng Z, Zhang M, Zhang Y (2016) Combining discrete and neural features for sequence labeling. In: Proceedings of the computational linguistics and intelligent text processing, pp 140–154. Springer

  63. Zhang C, Li Q, Song D (2019) Aspect-based sentiment classification with aspect-specific graph convolutional networks. In: Proceedings of the 2019 conference on empirical methods in natural language processing, pp 4568–4578

  64. Zhang S, Qin Y, Wen J, Wang X (2006) Word segmentation and named entity recognition for SIGHAN bakeoff3. In: Proceedings of the SIGHAN workshop on Chinese language processing, pp 158–161

  65. Zhang Y, Yang J (2018) Chinese NER using lattice LSTM. In: Proceedings of the annual meeting of the association for computational linguistics, pp 1554–1564

  66. Zhendong Dong QD (2003) Hownet—a hybrid language and knowledge resource. In: Proceedings of the natural language processing and knowledge engineering

  67. Zhou J, Qu W, Zhang F (2013) Chinese named entity recognition via joint identification and categorization. Chin J Electron 22(2):225–230

    Google Scholar 

  68. Zhu Y, Wang G (2019) CAN-NER: Convolutional Attention Network for Chinese Named Entity Recognition. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, pp 3384–3393

Download references

Funding

This work is supported by the science and technology research project of Jiang Xi Education Department under Grant Nos. GJJ201340, GJJ201302, GJJ201338, GJJ201341, the teaching reform research project of Jingdezhen Ceramic Institute under Grant No. TDJG-2020-02, the Key Research and Development Plan of Jiang Xi province under Grant No. 20202BBEL53020, and the College Student Innovation and Entrepreneurship Training Program under Grant Nos. 202110408027, S202110408067, S202110408107X, the Open Fund Project of Jiangxi Province Earthquake Prevention and Mitigation and Engineering Geological Disaster Detection Engineering Research Center under Grant No. SDGD202110, the Science and Technology project of Jingdezhen under Grant No. 20202GYZD013-001.

Author information

Authors and Affiliations

  1. Jingdezhen Ceramic University, Jingdezhen, China

    Yu Nie, Yilai Zhang, Yongkang Peng & Lisha Yang

Authors
  1. Yu Nie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yilai Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yongkang Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lisha Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

YN was conducting most of the experiments, and most of the article writing. YZ was responsible for part of the majority of the idea, as well as part of the writing and providing computational resources. YP was partially providing the idea, and conducting some of the experiments, e.g., data processing, analysis experiments, offering important suggestions. LY was conducting part of the experiments, and performing proofreading.

Corresponding authors

Correspondence to Yu Nie, Yilai Zhang or Yongkang Peng.

Ethics declarations

Conflict of interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Consent to participate

All authors have agreed the consent to participate in the work.

Consent for publication

All authors have checked the manuscript and have agreed to the publication.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nie, Y., Zhang, Y., Peng, Y. et al. Borrowing wisdom from world: modeling rich external knowledge for Chinese named entity recognition. Neural Comput & Applic 34, 4905–4922 (2022). https://doi.org/10.1007/s00521-021-06680-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06680-6

Keywords

Search

Navigation