Abstract
Relation extraction is a vital task in constructing large-scale knowledge graphs, aiming to identify factual relations between entities from plain texts and generate triples. However, it is inevitable that a large amount of noise will be generated and should be given special attention; otherwise, they will seriously downgrade the performance of knowledge reasoning. In this paper, we propose a visual analytics system that facilitates automatic extraction and interactive optimization of relations between entities, enabling users to refine these extraction results with low confidence. First, a triple-based embedding method is designed to provide an overview of the triples by capturing the semantic similarity between entities and relations. Then, the contextual information in the embedding space is utilized to evaluate the correctness of triples and infer more probable relations for correction. Finally, a visual analysis system integrating the above method and multiple coordinated views is developed, enabling the higher-quality data corrected by users to assist in achieving iterative optimization of the relation extraction model in an interpretable way. Case studies based on real-world datasets and expert interviews further demonstrate the effectiveness of the system for effective analysis and exploration of the knowledge graph relation extraction.
Graphical abstract
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bordes A, Usunier N, Garcia-Duran A, Weston J, Yakhnenko O (2013) Translating embeddings for modeling multi-relational data. In: Advances in neural information processing systems 26
Carlson A, Betteridge J, Kisiel B, Settles B, Hruschka E, Mitchell T (2010) Toward an architecture for never-ending language learning, vol 3
Cashman D, Xu S, Das S, Heimerl F, Liu C, Humayoun S, Gleicher M, Endert A, Chang R (2020) Cava: a visual analytics system for exploratory columnar data augmentation using knowledge graphs. IEEE Trans Vis Comput Gr. https://doi.org/10.1109/TVCG.2020.3030443
Chen W, Huang Z, Wu F, Zhu M, Maciejewski R (2018) VAUD: a visual analysis approach for exploring spatio-temporal urban data. IEEE Trans Vis Comput Gr 99:2636–2648
Dang T, Franz N, Ludäscher B, Forbes A (2015) Provenancematrix: a visualization tool for multi-taxonomy alignments. In: CEUR workshop proceedings vol 1456, pp 13–24
Fionda V, Pirrò G (2020) Learning triple embeddings from knowledge graphs. In: proceedings of the AAAI conference on artificial intelligence 34, pp 3874–3881
Han D, Pan J, Rusheng P, Zhou D, Cao N, He J, Xu M, Chen W (2022) iNet: visual analysis of irregular transition in multivariate dynamic networks. Front Comput Sci. https://doi.org/10.1007/s11704-020-0013-1
Hendrickx I, Kim S, Kozareva Z, Nakov P, Padó S, Pennacchiotti M, Romano L, Szpakowicz S (2010) Semeval-2010 task 8: Multi-way classification of semantic relations between pairs of nominals, pp 33–38
Henry Riche N, Fekete J-D, McGuffin M (2007) Nodetrix: a hybrid visualization of social networks. IEEE Trans Vis Comput Gr 13:1302–9. https://doi.org/10.1109/TVCG.2007.70582
Ji G, He S, Xu L, Liu K, Zhao J (2015) Knowledge graph embedding via dynamic mapping matrix, pp 687–696. https://doi.org/10.3115/v1/P15-1067
Kalinowski A, An Y (2022) Repurposing knowledge graph embeddings for triple representation via weak supervision. In: 2022 international conference on intelligent data science technologies and applications (IDSTA), IEEE, pp 129–137
Kratzwald B, Kunpeng G, Feuerriegel S, Diefenbach D (2020) Intkb: a verifiable interactive framework for knowledge base completion. https://doi.org/10.18653/v1/2020.coling-main.490
Li H, Wang Y, Zhang S, Song Y, Qu H (2021) KG4Vis: a knowledge graph-based approach for visualization recommendation. IEEE Trans Vis Comput Gr. https://doi.org/10.1109/TVCG.2021.3114863
Li Z, Wang X, Yang W, Wu J, Zhang Z, Liu Z, Sun M, Zhang H, Liu S (2022) A unified understanding of deep nlp models for text classification. IEEE Trans Vis Comput Gr 28(12):4980–4994. https://doi.org/10.1109/TVCG.2022.3184186
Lin Y, Liu Z, Sun M, Liu Y, Zhu X (2015) Learning entity and relation embeddings for knowledge graph completion. Proc AAAI 29:2181–2187. https://doi.org/10.1609/aaai.v29i1.9491
Liu S, Wang X, Chen J, Zhu J, Guo B (2015) Topicpanorama: a full picture of relevant topics. In: 2014 IEEE Conference on Visual Analytics Science and Technology, VAST 2014 - Proceedings 2014, pp 183–192 https://doi.org/10.1109/VAST.2014.7042494
Liu M, Shi J, Li Z, Li C, Zhu J, Liu S (2017) Towards better analysis of deep convolutional neural networks. IEEE Trans Vis Comput Gr 23(1):91–100
Lohfink A-P, Duque Anton S, Leitte H, Garth C (2021) Knowledge rocks: adding knowledge assistance to visualization systems. IEEE Trans Vis Comput Gr 28:1117
Ma C, Yang C, Yang F, Zhuang Y, Zhang Z, Jia H, Xie X (2018) Trajectory factory: tracklet cleaving and re-connection by deep siamese bi-gru for multiple object tracking. In: 2018 IEEE international conference on multimedia and Expo (ICME), pp 1–6. https://doi.org/10.1109/ICME.2018.8486454
Miwa M, Bansal M (2016) End-to-end relation extraction using lstms on sequences and tree structures, pp 1105–1116. https://doi.org/10.18653/v1/P16-1105
Nickel M, Tresp V, Kriegel H-P et al (2011) A three-way model for collective learning on multi-relational data. In: Icml 11, pp 3104482–3104584
Nickel M, Rosasco L, Poggio T (2015) Holographic embeddings of knowledge graphs. In: proceedings of the AAAI conference on artificial intelligence 30https://doi.org/10.1609/aaai.v30i1.10314
Peng Y, Fan X, Chen R, Yu Z, Liu S, Chen Y, Ying Z, Zhou F (2023) Visual abstraction of dynamic network via improved multi-class blue noise sampling. Front Comput Sci. https://doi.org/10.1007/s11704-021-0609-0
Schutz A, Buitelaar P (2005) Relext: a tool for relation extraction from text in ontology extension, pp 593–606. https://doi.org/10.1007/11574620_43
Sheng S, Zhou P, Wu X (2019) CEPV: a tree structure information extraction and visualization tool for big knowledge graph, pp 221–228. https://doi.org/10.1109/ICBK.2019.00037
Shinyama Y, Sekine S (2006). Preemptive information extraction using unrestricted relation discovery. https://doi.org/10.3115/1220835.1220874
Sinclair G, Thillainadarajah I, Meyer B, Samano V, Sivasupramaniam S, Adams L, Willighagen E, Richard A, Walker M, Williams A (2022) Wikipedia on the comptox chemicals dashboard: connecting resources to enrich public chemical data. J Chem Inf Model. https://doi.org/10.1021/acs.jcim.2c00886
Sun K, Liu Y, Guo Z, Wang C (2016) EduVis: visualization for education knowledge graph based on web data, pp 138–139. https://doi.org/10.1145/2968220.2968227
Trouillon T, Welbl J, Riedel S, Gaussier É, Bouchard G (2016) Complex embeddings for simple link prediction. In: international conference on machine learning, PMLR, pp 2071–2080
Wang Z, Zhang J, Feng J, Chen Z (2014) Knowledge graph embedding by translating on hyperplanes. In: proceedings of the AAAI conference on artificial intelligence, 28.https://doi.org/10.1609/aaai.v28i1.8870
Wang Q, Mao Z, Wang B, Guo L (2017) Knowledge graph embedding: a survey of approaches and applications. IEEE Trans Knowl Data Eng 29(12):2724–2743
Wang X, Wu Z, Huang W, Wei Y, Huang Z, Xu M, Chen W (2023) VIS+AI: integrating visualization with artificial intelligence for efficient data analysis. Front Comput Sci 17(6):1
Weihua Y, Dong X (2021) Visual analysis of industrial knowledge graph research based on citespace, pp 297–300. https://doi.org/10.1109/CMMNO53328.2021.9467534
Xi J, Ye L, Huang Q, Li X (2021) Tolerating data missing in breast cancer diagnosis from clinical ultrasound reports via knowledge graph inference, pp 3756–3764. https://doi.org/10.1145/3447548.3467106
Xia J-z, Zhang Y-h, Ye H, Wang Y, Jiang G, Zhao Y, Xie C, Kui X-y, Liao S-h, Wang W-p (2020) Supoolvisor: a visual analytics system for mining pool surveillance. Front Inf Technol Electron Eng 21(4):507–523. https://doi.org/10.1631/FITEE.1900532
Xia J, Huang L, Lin W, Zhao X, Wu J, Chen Y, Zhao Y, Chen W (2022) Interactive visual cluster analysis by contrastive dimensionality reduction. IEEE Trans Vis Comput Gr 29(1):734–744
Xia J, Huang L, Lin W, Zhao X, Wu J, Chen Y, Zhao Y, Chen W (2023) Interactive visual cluster analysis by contrastive dimensionality reduction. IEEE Trans Vis Comput Gr 29(1):734–744. https://doi.org/10.1109/TVCG.2022.3209423
Xiao J, Zhou Z (2020) Chapter-level entity relationship extraction method based on joint learning, pp 75–78. https://doi.org/10.1109/IHMSC49165.2020.00025
Xiong C, Power R, Callan J (2017) Explicit semantic ranking for academic search via knowledge graph embedding, pp 1271–1279. https://doi.org/10.1145/3038912.3052558
Xu K, Feng Y, Huang S, Zhao D (2015) Semantic relation classification via convolutional neural networks with simple negative sampling https://doi.org/10.18653/v1/D15-1062
Yang B, Yih W-t, He X, Gao J, Deng L (2014) Embedding entities and relations for learning and inference in knowledge bases. arXiv preprint arXiv:1412.6575
Yang W, Liu M, Wang Z, Liu S (2024) Foundation models meet visualizations: challenges and opportunities. Computational Visual Media. arxiv: 2310.05771
Ying Z, Luhao G, Huixuan X, Bai G, Zhang Z, Wei Q, Lin Y, Liu Y, Zhou F (2022) Astf: visual abstractions of time-varying patterns in radio signals. IEEE Trans Vis Comput Gr. https://doi.org/10.48550/arXiv.2209.15223d
Yuyu Z, Dai H, Kozareva Z, Smola A, Song L (2017) Variational reasoning for question answering with knowledge graph. In: proceedings of the AAAI conference on artificial intelligence, 32
Zeng D, Liu K, Lai S, Zhou G, Zhao J (2014) Relation classification via convolutional deep neural network. In: proceedings of COLING 2014, the 25th international conference on computational linguistics: technical papers, pp 2335–2344
Zhang Y, Qi P, Manning C (2018) Graph convolution over pruned dependency trees improves relation extraction, pp 2205–2215. https://doi.org/10.18653/v1/D18-1244
Zhang N, Deng S, Sun Z, Wang G, Chen X, Zhang W, Chen H (2019) Long-tail relation extraction via knowledge graph embeddings and graph convolution networks
Zhang Z, Cai J, Zhang Y, Wang J (2020) Learning hierarchy-aware knowledge graph embeddings for link prediction. In: proceedings of the AAAI conference on artificial intelligence 34, pp 3065–3072
Zheng S, Wang F, Bao H, Hao Y, Zhou P, Xu B (2017) Joint extraction of entities and relations based on a novel tagging scheme
Zhou P, Shi W, Tian J, Qi Z, Li B, Hao H, Xu B (2016) Attention-based bidirectional long short-term memory networks for relation classification, pp 207–212. https://doi.org/10.18653/v1/P16-2034
Zhou Z, Shi C, Shen X, Cai L, Wang H, Liu Y, Ying Z, Chen W (2020a) Context-aware sampling of large networks via graph representation learning. IEEE Trans Vis Comput Gr. https://doi.org/10.1109/TVCG.2020.3030440
Zhou Z, Zhang X, Yang Z, Chen Y, Liu Y, Wen J, Chen B, Ying Z, Chen W (2020b) Visual abstraction of geographical point data with spatial autocorrelations, pp 60–71. https://doi.org/10.1109/VAST50239.2020.00011
Zhou Z, Sun L, Yu W, Liu Y, Xiang Z, Wang Y, Chen W (2022) iMGC: interactive multiple graph clustering with constrained Laplacian rank. IEEE Trans Hum Mach Syst. https://doi.org/10.1109/THMS.2022.3227181
Zhou Z, Zheng F, Wen J, Chen Y, Li X, Liu Y, Wang Y, Chen W (2023) A user-driven sampling model for large-scale geographical point data visualization via convolutional neural networks. IEEE Trans Hum Mach Syst. https://doi.org/10.1109/THMS.2023.3296692
Acknowledgments
This work was supported in part by the National Natural Science Foundation of China (U22A2032), National Statistical Science Research Project (No.2022LY099), Zhejiang Laboratory Open Research Project (No.K2022KG0AB01), Public Welfare Plan Research Project of Zhejiang Provincial Science and Technology Department (No. LTGG23H260003), Zhejiang Provincial Natural Science Foundation of China under Grant (No.LTGG24F020006), Open Project Program of the State Key Laboratory of CAD&CG of Zhejiang University (No.A2301), and Zhejiang Statistical Science Research Project. Finally, we would like to express our sincere gratitude to Zhengkai Xiao, Ke Lu, and Zhenyi Yang for their invaluable assistance in this paper, as they have offered us numerous insightful opinions and guidance.
Author information
Authors and Affiliations
Corresponding authors
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Supplementary file1 (MP4 13980 kb)
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Liu, Y., Ma, Y., Zhang, Y. et al. Interactive optimization of relation extraction via knowledge graph representation learning. J Vis 27, 197–213 (2024). https://doi.org/10.1007/s12650-024-00955-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12650-024-00955-5