Abstract
Relations in knowledge graphs often exhibit multiple relation patterns. Various knowledge graph embedding methods have been proposed to modelling properties in relation patterns. However, relations with a certain relation pattern actually only account for a small proportion in the knowledge graph. Relations with no explicit relation patterns also show complicated properties which is rarely studied. To this end, we argue that a property of a relation should either be global or be partial, and propose an Attention-based Learning framework for Multi-relation Patterns (ALMP) for expressing complex properties of relations. ALMP adopts a set of affine transformations to express corresponding global relation properties. Furthermore, ALMP utilizes a module of attention mechanism to integrate the representations. Experimental results show that ALMP outperforms baseline models on the link prediction task.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
Note that relational rotation can model symmetric pattern only when the relational rotation phase is \(n\pi \), \((n=0,1,2,\dots )\). While reflection is more general and straightforward for modeling symmetric pattern.
- 3.
A shallow encoder in KG embedding can be viewed as a lookup function that finds the hidden representation corresponding to an entity or a relation given its index [9].
References
Abboud, R., İsmail İ.C., Lukasiewicz, T., Salvatori, T.: BoxE: a box embedding model for knowledge base completion (2020)
Yang, B., Yih, W. T., He, X., Gao, J., Deng, L.: Embedding entities and relations for learning and inference in knowledge bases. In: ICLR, pp. 1–13 (2015)
Balazevic, I., Allen, C., Hospedales, T.: Tucker: tensor factorization for knowledge graph completion. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (2019). https://doi.org/10.18653/v1/d19-1522
Battaglia, P.W., et al.: Relational inductive biases, deep learning, and graph networks. CoRR (2018)
Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. Adv. Neural Inf. Process. Syst. 26, 2787–2795 (2013)
Chami, I., Wolf, A., Juan, D.C., Sala, F., Ravi, S., Ré, C.: Low-dimensional hyperbolic knowledge graph embeddings. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 6901–6914. Association for Computational Linguistics (Jul 2020)
Chao, L., He, J., Wang, T., Chu, W.: PairRE: knowledge graph embeddings via paired relation vectors. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, vol. 1: Long Papers, pp. 4360–4369. Association for Computational Linguistics, Online (2021). https://aclanthology.org/2021.acl-long.336
Bollacker, K., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: a collaboratively created graph database for structuring human knowledge. In: SIGMOD, pp. 1247–1250 (2008)
Kazemi, S.M., et al.: Representation learning for dynamic graphs: a survey (2020)
Lacroix, T., Usunier, N., Obozinski, G.: Canonical tensor decomposition for knowledge base completion (2018)
Mahdisoltani, F., Biega, J., Suchanek, F.: Yago3: a knowledge base from multilingual wikipedias. In: Proceedings of CIDR 2015 (2015)
Miller, G.A.: Wordnet: a lexical database for English. Commun. ACM 38(11), 39–41 (1995)
Nathani, D., Chauhan, J., Sharma, C., Kaul, M.: Learning attention-based embeddings for relation prediction in knowledge graphs. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4710–4723 (2019)
Song, T., Luo, J., Huang, L.: Rot-pro: modeling transitivity by projection in knowledge graph embedding. In: Proceedings of the Thirty-Fifth Annual Conference on Advances in Neural Information Processing Systems (NeurIPS) (2021)
Suchanek, F.M., Kasneci, G., Weikum, G.: Yago: a core of semantic knowledge. In: Proceedings of the 16th International Conference on World Wide Web, pp. 697–706 (2007)
Sun, Z., Deng, Z.H., Nie, J.Y., Tang, J.: Rotate: knowledge graph embedding by relational rotation in complex space. In: International Conference on Learning Representations (2019)
Trouillon, T., Welbl, J., Riedel, S., Gaussier, E., Bouchard, G.: Complex embeddings for simple link prediction. In: Proceedings of 33rd International Conference on Machine Learning, pp. 2071–2080 (2016)
Dettmers, T., Minervini, P., Stenetorp, P., Riedel, S.: Convolutional 2D knowledge graph embeddings. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence (2018)
Toutanova, K., Chen, D.: Observed versus latent features for knowledge base and text inference. In: Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality, pp. 57–66 (2015)
Valenza, R.: Linear algebra: an introduction to abstract mathematics. In: Undergraduate Texts in Mathematics. Springer, New York (2012). https://doi.org/10.1007/978-1-4612-0901-0
Wang, R., Li, B., Hu, S., Du, W., Zhang, M.: Knowledge graph embedding via graph attenuated attention networks. IEEE Access 8, 5212–5224 (2020)
Wang, Z., Zhang, J., Feng, J., Chen, Z.: Knowledge graph embedding by translating on hyperplanes. In: AAAI Conference on Artificial Intelligence (2014)
Lin, Y., Liu, Z., Sun, M., Liu, Y., Zhu, X.: Learning entity and relation embeddings for knowledge graph completion. In: Proceedings of the 29th AAAI Conference on Artificial Intelligence, pp. 2181–2187 (2015)
Zhang, S., Tay, Y., Yao, L., Liu, Q.: Quaternion knowledge graph embedding. In: Advances in Neural Information Processing Systems, pp. 2731–2741 (2019)
Zhang, W., et al.: Iteratively learning embeddings and rules for knowledge graph reasoning (2019)
Acknowledgement
This work was supported by the National Key R &D Program of China (Grant No. 2021ZD0112901).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Song, T., Luo, J. (2022). Attention-based Learning for Multiple Relation Patterns in Knowledge Graph Embedding. In: Memmi, G., Yang, B., Kong, L., Zhang, T., Qiu, M. (eds) Knowledge Science, Engineering and Management. KSEM 2022. Lecture Notes in Computer Science(), vol 13368. Springer, Cham. https://doi.org/10.1007/978-3-031-10983-6_50
Download citation
DOI: https://doi.org/10.1007/978-3-031-10983-6_50
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-10982-9
Online ISBN: 978-3-031-10983-6
eBook Packages: Computer ScienceComputer Science (R0)