Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Span-based joint entity and relation extraction augmented with sequence tagging mechanism

  • Research Paper
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

Span-based joint extraction simultaneously conducts named entity recognition (NER) and relation extraction (RE) in a text span form. However, since previous span-based models rely on span-level classifications, they cannot benefit from token-level label information, which has been proven advantageous for the task. In this paper, we propose a sequence tagging augmented span-based network (STSN), a span-based joint model that can make use of token-level label information. In STSN, we construct a core neural architecture by deep stacking multiple attention layers, each of which consists of three basic attention units. On the one hand, the core architecture enables our model to learn token-level label information via the sequence tagging mechanism and then uses the information in the span-based joint extraction; on the other hand, it establishes a bi-directional information interaction between NER and RE. Experimental results on three benchmark datasets show that STSN consistently outperforms the strongest baselines in terms of F1, creating new state-of-the-art results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  1. Li Q, Ji H. Incremental joint extraction of entity mentions and relations. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, Baltimore, 2014. 402–412

  2. Miwa M, Bansal M. End-to-end relation extraction using LSTMs on sequences and tree structures. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, 2016. 1105–1116

  3. Katiyar A, Cardie C. Going out on a limb: joint extraction of entity mentions and relations without dependency trees. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, 2017. 917–928

  4. Ye W, Li B, Xie R, et al. Exploiting entity BIO tag embeddings and multi-task learning for relation extraction with imbalanced data. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, 2019. 1351–1360

  5. Lin Y, Ji H, Huang F, et al. A joint neural model for information extraction with global features. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020. 7999–8009

  6. Luan Y, He L, Ostendorf M, et al. Multi-task identification of entities, relations, and coreference for scientific knowledge graph construction. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, 2018. 3219–3232

  7. Dixit K, Al-Onaizan Y. Span-level model for relation extraction. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, 2019. 5308–5314

  8. Eberts M, Ulges A. Span-based joint entity and relation extraction with transformer pre-training. In: Proceedings of the 24th European Conference on Artificial Intelligence, Santiago de Compostela, 2020. 1–8

  9. Ji B, Yu J, Li S, et al. Span-based joint entity and relation extraction with attention-based span-specific and contextual semantic representations. In: Proceedings of the 28th International Conference on Computational Linguistics, 2020. 88–99

  10. Zhong Z, Chen D. A frustratingly easy approach for entity and relation extraction. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021. 50–61

  11. Luan Y, Wadden D, He L, et al. A general framework for information extraction using dynamic span graphs. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, 2019. 3036–3046

  12. Wadden D, Wennberg U, Luan Y, et al. Entity, relation, and event extraction with contextualized span representations. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, 2019. 5784–5789

  13. Yao Y, Ye D, Li P, et al. DocRED: a large-scale document-level relation extraction dataset. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, 2019. 764–777

  14. Zhang Y, Zhong V, Chen D, et al. Position-aware attention and supervised data improve slot filling. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, 2017. 35–45

  15. Riedel S, Yao L, McCallum A. Modeling relations and their mentions without labeled text. In: Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Berlin, 2010. 148–163

  16. Zeng X, Zeng D, He S, et al. Extracting relational facts by an end-to-end neural model with copy mechanism. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, 2018. 506–514

  17. Hendrickx I, Kim S, Kozareva Z, et al. SemEval-2010 task 8: multi-way classification of semantic relations between pairs of nominals. In: Proceedings of the 5th International Workshop on Semantic Evaluation, Uppsala, 2010. 33–38

  18. Roth D, Yih W. A linear programming formulation for global inference in natural language tasks. In: Proceedings of the 8th Conference on Computational Natural Language Learning at HLT-NAACL, Boston, 2004. 1–8

  19. Gurulingappa H, Rajput A M, Roberts A, et al. Development of a benchmark corpus to support the automatic extraction of drug-related adverse effects from medical case reports. J Biomed Inf, 2012, 45: 885–892

    Article  Google Scholar 

  20. Bekoulis G, Deleu J, Demeester T, et al. Joint entity recognition and relation extraction as a multi-head selection problem. Expert Syst Appl, 2018, 114: 34–45

    Article  Google Scholar 

  21. Zhao S, Hu M, Cai Z, et al. Modeling dense cross-modal interactions for joint entity-relation extraction. In: Proceedings of the 29th International Joint Conference on Artificial Intelligence, 2020. 4032–4038

  22. Lee K, He L, Zettlemoyer L. Higher-order coreference resolution with coarse-to-fine inference. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, 2018. 687–692

  23. He L, Lee K, Levy O, et al. Jointly predicting predicates and arguments in neural semantic role labeling. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, 2018. 364–369

  24. Peters M, Neumann M, Iyyer M, et al. Deep contextualized word representations. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, 2018. 2227–2237

  25. Devlin J, Chang M, Lee K, et al. BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, 2019. 4171–4186

  26. Lan Z, Chen M, Goodman S, et al. ALBERT: a lite BERT for self-supervised learning of language representations. In: Proceedings of the 8th International Conference on Learning Representations, 2020. 1–17

  27. Wu Y, Schuster M, Chen Z, et al. Google’s neural machine translation system: bridging the gap between human and machine translation. 2016. ArXiv:1609.08144

  28. Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need. In: Proceedings of the 31st Conference on Neural Information Processing Systems, Long Beach, 2017. 5998–6008

  29. Doddington G, Mitchell A, Przybocki M, et al. The automatic content extraction (ACE) program — tasks, data, and evaluation. In: Proceedings of the 4th International Conference on Language Resources and Evaluation, Lisbon, 2004. 837–840

  30. Wang J, Lu W. Two are better than one: joint entity and relation extraction with table-sequence encoders. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, 2020. 1706–1721

  31. Sun C, Wu Y, Lan M, et al. Extracting entities and relations with joint minimum risk training. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, 2018. 2256–2265

  32. Li X, Yin F, Sun Z, et al. Entity-relation extraction as multi-turn question answering. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, 2019. 1340–1350

  33. Shen Y, Ma X, Tang Y, et al. A trigger-sense memory flow framework for joint entity and relation extraction. In: Proceedings of the Web Conference 2021, 2021. 1704–1715

  34. Ren L, Sun C, Ji H, et al. HySPA: hybrid span generation for scalable text-to-graph extraction. In: Findings of the Association for Computational Linguistics: ACL-IJCNLP, 2021. 4066–4078

  35. Nguyen D Q, Verspoor K. End-to-end neural relation extraction using deep biaffine attention. In: Proceedings of the 41st European Conference on Information Retrieval, Cologne, 2019. 729–738

  36. Miwa M, Sasaki Y. Modeling joint entity and relation extraction with table representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, 2014. 1858–1869

  37. Zhang M, Zhang Y, Fu G. End-to-end neural relation extraction with global optimization. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, 2017. 1730–1740

  38. Huguet P, Navigli R. REBEL: relation extraction by end-to-end language generation. In: Findings of the 2021 Conference on Empirical Methods in Natural Language Processing, Punta Cana, 2021. 2370–2381

  39. Yan Z, Zhang C, Fu J, et al. A partition filter network for joint entity and relation extraction. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Punta Cana, 2021. 185–197

  40. Lai T, Ji H, Zhai C, et al. Joint biomedical entity and relation extraction with knowledge-enhanced collective inference. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, 2021. 6248–6260

  41. Li F, Zhang Y, Zhang M, et al. Joint models for extracting adverse drug events from biomedical text. In: Proceedings of the 25th International Joint Conference on Artificial Intelligence, New York, 2016. 2838–2844

  42. Li F, Zhang M, Fu G, et al. A neural joint model for entity and relation extraction from biomedical text. BMC Bioinf, 2017, 18: 198

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by Hunan Provincial Natural Science Foundation (Grant Nos. 2022JJ30668, 2022JJ30046).

Author information

Authors and Affiliations

  1. College of Computer, National University of Defense Technology, Changsha, 410073, China

    Bin Ji, Shasha Li, Hao Xu, Jie Yu, Jun Ma, Huijun Liu & Jing Yang

Authors
  1. Bin Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shasha Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hao Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jie Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jun Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Huijun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jing Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Shasha Li, Hao Xu, Jie Yu or Huijun Liu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ji, B., Li, S., Xu, H. et al. Span-based joint entity and relation extraction augmented with sequence tagging mechanism. Sci. China Inf. Sci. 67, 152105 (2024). https://doi.org/10.1007/s11432-022-3608-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11432-022-3608-y

Keywords

Search

Navigation