Abstract
Knowledge representation learning (KRL) aims to project entities and relations in knowledge graphs (KGs) to densely distributed embedding space. As the knowledge base expands, we are often presented with zero-shot entities, often with textual descriptions. Although many closed-world KRL methods have been proposed, most of them focus on connections between entities in the existing KGs. Therefore, they cannot handle zero-shot entities well, resulting in the inability of bringing zero-shot entities to existing KGs. To address this issue, this paper proposes ASKRL, a straightforward yet efficient open-world knowledge representation learning framework. ASKRL learns representations of entities and relations in both structured and semantic spaces, and subsequently aligns the semantic space with the structured space. To begin with, ASKRL employs the off-the-shelf KRL models to derive entity and relation embeddings in the structured embedding space. Afterward, a Transformer-based encoder is applied to obtain contextualized representations of existing entities and relations in semantic space. To introduce structure knowledge of KG into the contextualized representations, ASKRL aligns semantic embedding space to structured embedding space from the perspective of common properties (i.e., angle and length). Additionally, it aligns the output distribution of the score function between the two spaces. To further learn representations of zero-shot entities effectively, a sophisticated three-stage optimization strategy is devised in the training phase. In the inference phase, representations of zero-shot entities can be directly derived from the Transformer-based encoder. ASKRL is plug-and-play, enabling off-the-shelf closed-world KRL models to handle the open-world KGs. Extensive experiments demonstrate that ASKRL significantly outperforms strong baselines in open-world datasets, and the results illuminate that ASKRL is simple and efficient in modeling zero-shot entities.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Auer, S., Bizer, C., Kobilarov, G., et al.: Dbpedia: a nucleus for a web of open data. In: Proceedings of the 6th International Semantic Web Conference (2007)
Bollacker, K., Evans, C., Paritosh, P., et al.: Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the 27th ACM SIGMOD International Conference (2008)
Bordes, A., Usunier, N., García-Durán, A., et al.: Translating embeddings for modeling multi-relational data. In: Proceedings of the 27th Neural Information Processing Systems (2013)
Daza, D., Cochez, M., Groth, P.: Inductive entity representations from text via link prediction. In: Proceedings of the Web Conference 2021 (2021)
Devlin, J., Chang, M., Lee, K., et al.: Bert: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (2019)
Dong, X.L., Gabrilovich, E., Heitz, G., et al.: Knowledge vault: a web-scale approach to probabilistic knowledge fusion. In: Proceedings of the 20th ACM SIGKDD International Conference (2014)
Galárraga, L., Teflioudi, C., Hose, K., Suchanek, F.M.: Fast rule mining in ontological knowledge bases with AMIE+. VLDB J. 24(6), 707–730 (2015)
Galárraga, L.A., Teflioudi, C., Hose, K., Suchanek, F.: Amie: association rule mining under incomplete evidence in ontological knowledge bases. In: Proceedings of the Web Conference 2013 (2013)
Hogan, A., Blomqvist, E., Cochez, M., et al.: Knowledge graphs. ACM Comput. Surv. 54(4), 1–37 (2021)
Li, X., Luo, X., Dong, C., et al.: TDEER: an efficient translating decoding schema for joint extraction of entities and relations. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (2021)
Li, Z., Liu, H., Zhang, Z., Liu, T., Xiong, N.N.: Learning knowledge graph embedding with heterogeneous relation attention networks. IEEE Trans. Neural Netw. Learn. Syst. 33(8), 3961–3973 (2022)
Liu, Y., et al.: Roberta: a robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692 (2019)
Liu, Y., Wang, P., Li, Y., et al.: Aprile: attention with pseudo residual connection for knowledge graph embedding. In: Proceedings of the 28th International Conference on Computational Linguistics (2020)
Mohammed, S., Shi, P., Lin, J.: Strong baselines for simple question answering over knowledge graphs with and without neural networks. In: Proceedings of the 16th North American Chapter of the Association for Computational Linguistics (2018)
Park, W., Kim, D., Lu, Y., Cho, M.: Relational knowledge distillation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2019)
Reiter, R.: On closed world data bases. In: Webber, B.L., Nilsson, N.J. (eds.) Readings in Artificial Intelligence, pp. 119–140. Morgan Kaufmann (1981)
Sadeghian, A., Armandpour, M., Ding, P., Wang, D.Z.: Drum: end-to-end differentiable rule mining on knowledge graphs. In: Proceedings of the 33th Neural Information Processing Systems (2019)
Shah, H., Villmow, J., Ulges, A., et al.: An open-world extension to knowledge graph completion models. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence (2019)
Shi, B., Weninger, T.: Open-world knowledge graph completion. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence (2018)
Suchanek, F.M., Kasneci, G., Weikum, G.: Yago: a core of semantic knowledge. In: Proceedings of the 16th International Conference on World Wide Web (2007)
Sun, Z., Deng, Z., Nie, J., et al.: Rotate: knowledge graph embedding by relational rotation in complex space. In: Proceedings of the 7th International Conference on Learning Representations (2019)
Teru, K.K., Denis, E.G., Hamilton, W.L.: Inductive relation prediction by subgraph reasoning. In: Proceedings of the 37th International Conference on Machine Learning (2020)
Toutanova, K., Chen, D., Pantel, P., et al.: Representing text for joint embedding of text and knowledge bases. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (2015)
Trouillon, T., Welbl, J., Riedel, S., et al.: Complex embeddings for simple link prediction. In: Proceedings of the 33nd International Conference on Machine Learning (2016)
Wang, B., Shen, T., Long, G., Zhou, T., Wang, Y., Chang, Y.: Structure-augmented text representation learning for efficient knowledge graph completion. In: Proceedings of the Web Conference 2021 (2021)
Wang, J., Wang, Z., Zhang, D., et al.: Combining knowledge with deep convolutional neural networks for short text classification. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence (2017)
Wang, X., He, X., Cao, Y., et al.: KGAT: knowledge graph attention network for recommendation. In: Proceedings of the 25th ACM SIGKDD International Conference (2019)
Wang, X., et al.: Kepler: a unified model for knowledge embedding and pre-trained language representation. Trans. Assoc. Comput. Linguist. 9, 176–194 (2021)
Wang, Y., Xiao, W.D., Tan, Z., Zhao, X.: Caps-OWKG: a capsule network model for open-world knowledge graph. Int. J. Mach. Learn. Cybern. 12, 1627–1637 (2021)
Wei, Z., Su, J., Wang, Y., Tian, Y., et al.: A novel cascade binary tagging framework for relational triple extraction. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (2020)
Xie, R., Liu, Z., Jia, J., et al.: Representation learning of knowledge graphs with entity descriptions. In: Proceedings of the 30th AAAI Conference on Artificial Intelligence (2016)
Xu, X., Zhang, P., He, Y., Chao, C., Yan, C.: Subgraph neighboring relations infomax for inductive link prediction on knowledge graphs. In: Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence (2022)
Yamada, I., Shindo, H., Takeda, H., Takefuji, Y.: Joint learning of the embedding of words and entities for named entity disambiguation. In: Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning (2016)
Yang, B., Yih, W., He, X., et al.: Embedding entities and relations for learning and inference in knowledge bases. In: Proceedings of the 3rd International Conference on Learning Representations (2015)
Yang, F., Yang, Z., Cohen, W.W.: Differentiable learning of logical rules for knowledge base reasoning. In: Proceedings of the 31th Neural Information Processing Systems (2017)
Zhong, Z., Chen, D.: A frustratingly easy approach for entity and relation extraction. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics (2021)
Zhou, Y., Shi, S., Huang, H.: Weighted aggregator for the open-world knowledge graph completion. In: Proceedings of the 6th International Conference of Pioneering Computer Scientists (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Shang, Z., Wang, P., Liu, Y., Liu, J., Ke, W. (2023). ASKRL: An Aligned-Spatial Knowledge Representation Learning Framework for Open-World Knowledge Graph. In: Payne, T.R., et al. The Semantic Web – ISWC 2023. ISWC 2023. Lecture Notes in Computer Science, vol 14265. Springer, Cham. https://doi.org/10.1007/978-3-031-47240-4_6
Download citation
DOI: https://doi.org/10.1007/978-3-031-47240-4_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-47239-8
Online ISBN: 978-3-031-47240-4
eBook Packages: Computer ScienceComputer Science (R0)