research-article

Enhancing the convolution-based knowledge graph embeddings by increasing dimension-wise interactions

Authors:

Xinli HuangAuthors Info & Claims

Volume 146, Issue C

https://doi.org/10.1016/j.datak.2023.102184

Published: 01 July 2023 Publication History

Abstract

Knowledge graph embedding learns distributed low-dimensional representations for the elements in knowledge graphs, so that knowledge can be conveniently integrated into various tasks and smart systems. Recently, convolutional neural network has been introduced to embedding technique and obtained impressive achievements in link prediction task. ConvKB, a recently proposed method, captured the global dimension-wise interactions in facts with the convolutional filters. However, ConvKB failed to learn the local interactions between the entity and relation embedding. Moreover, rich interactions among feature maps are neglected in the existing convolutional embedding models. In this paper, based on ConvKB, we propose ConvD which models the local relationships in facts and integrates the cross-channel information based on the dimension-wise interactions to further improve the performance. From the experimental results, ConvD obtains scores that are 96% and 5% better than ConvKB on MRR and Hits@10 in the link prediction task. Furthermore, ConvD surpassed state-of-the-art baselines on WN18RR and achieved competitive results on FB15k-237 respectively.

References

[1]

K. Bollacker, C. Evans, P. Paritosh, T. Sturge, J. Taylor, Freebase: a collaboratively created graph database for structuring human knowledge, in: Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, 2008, pp. 1247–1250.

[2]

Lehmann J., Isele R., Jakob M., Jentzsch A., Kontokostas D., Mendes P.N., Hellmann S., Morsey M., Van Kleef P., Auer S., et al., Dbpedia–a large-scale, multilingual knowledge base extracted from wikipedia, Semant. Web 6 (2) (2015) 167–195.

[3]

Vrandečić D., Krötzsch M., Wikidata: A free collaborative knowledgebase, Commun. ACM 57 (10) (2014) 78–85.

Digital Library

[4]

J. Berant, A. Chou, R. Frostig, P. Liang, Semantic parsing on freebase from question-answer pairs, in: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, 2013, pp. 1533–1544.

[5]

Bordes A., Weston J., Usunier N., Open question answering with weakly supervised embedding models, in: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Springer, 2014, pp. 165–180.

[6]

J. Daiber, M. Jakob, C. Hokamp, P.N. Mendes, Improving efficiency and accuracy in multilingual entity extraction, in: Proceedings of the 9th International Conference on Semantic Systems, 2013, pp. 121–124.

[7]

Wang Q., Mao Z., Wang B., Guo L., Knowledge graph embedding: A survey of approaches and applications, IEEE Trans. Knowl. Data Eng. 29 (12) (2017) 2724–2743.

[8]

Bordes A., Usunier N., Garcia-Duran A., Weston J., Yakhnenko O., Translating embeddings for modeling multi-relational data, in: Burges C.J.C., Bottou L., Welling M., Ghahramani Z., Weinberger K.Q. (Eds.), Advances in Neural Information Processing Systems 26, Curran Associates, Inc., 2013, pp. 2787–2795.

[9]

Ji G., He S., Xu L., Liu K., Zhao J., Knowledge graph embedding via dynamic mapping matrix, in: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Association for Computational Linguistics, 2015, pp. 687–696.

[10]

Y. Lin, Z. Liu, M. Sun, Y. Liu, X. Zhu, Learning entity and relation embeddings for knowledge graph completion, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 29. No. 1, 2015.

[11]

B. Yang, W. tau Yih, X. He, J. Gao, L. Deng, Embedding Entities and Relations for Learning and Inference in Knowledge Bases, in: 3rd International Conference on Learning Representations, 2015.

[12]

Trouillon T., Welbl J., Riedel S., Gaussier É., Bouchard G., Complex embeddings for simple link prediction, in: International Conference on Machine Learning, PMLR, 2016, pp. 2071–2080.

[13]

Socher R., Chen D., Manning C.D., Ng A., Reasoning with neural tensor networks for knowledge base completion, in: Burges C., Bottou L., Welling M., Ghahramani Z., Weinberger K. (Eds.), Advances in Neural Information Processing Systems, Vol. 26, Curran Associates, Inc., 2013, pp. 926–934.

[14]

T. Dettmers, P. Minervini, P. Stenetorp, S. Riedel, Convolutional 2D Knowledge Graph Embeddings, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32, No. 1, 2018.

[15]

Nguyen D.Q., Nguyen T.D., Nguyen D.Q., Phung D., A novel embedding model for knowledge base completion based on convolutional neural network, in: NAACL HLT 2018: 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies-Proceedings of the Conference, Association for Computational Linguistics, 2018, pp. 327–333.

[16]

Z. Wang, J. Zhang, J. Feng, Z. Chen, Knowledge graph embedding by translating on hyperplanes, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 28, No. 1, 2014.

[17]

Toutanova K., Chen D., Pantel P., Poon H., Choudhury P., Gamon M., Representing text for joint embedding of text and knowledge bases, in: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Lisbon, Portugal, 2015, pp. 1499–1509.

[18]

He S., Liu K., Ji G., Zhao J., Learning to represent knowledge graphs with Gaussian embedding, in: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, CIKM ’15, Association for Computing Machinery, New York, NY, USA, 2015, pp. 623–632.

[19]

H. Xiao, M. Huang, X. Zhu, From one point to a manifold: knowledge graph embedding for precise link prediction, in: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, 2016, pp. 1315–1321.

[20]

Z. Sun, Z.-H. Deng, J.-Y. Nie, J. Tang, RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space, in: International Conference on Learning Representations, 2018.

[21]

M. Nickel, V. Tresp, H.-P. Kriegel, A Three-Way Model for Collective Learning on Multi-Relational Data, in: Proceedings of the 28th International Conference on Machine Learning, 2011, pp. 809–816.

[22]

T. Trouillon, J. Welbl, S. Riedel, É. Gaussier, G. Bouchard, Complex embeddings for simple link prediction, in: Proceedings of the 33nd International Conference on Machine Learning, ICML, 2016, pp. 2071–2080.

[23]

X. Dong, E. Gabrilovich, G. Heitz, W. Horn, N. Lao, K. Murphy, T. Strohmann, S. Sun, W. Zhang, Knowledge vault: A web-scale approach to probabilistic knowledge fusion, in: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2014, pp. 601–610.

[24]

C. Shang, Y. Tang, J. Huang, J. Bi, X. He, B. Zhou, End-to-end structure-aware convolutional networks for knowledge base completion, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33, ISBN: 2374-3468, 2019, pp. 3060–3067.

[25]

S. Vashishth, S. Sanyal, V. Nitin, N. Agrawal, P.P. Talukdar, InteractE: Improving Convolution-Based Knowledge Graph Embeddings by Increasing Feature Interactions, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2020, pp. 3009–3016.

[26]

Z. Xie, G. Zhou, J. Liu, X. Huang, ReInceptionE: Relation-aware inception network with joint local-global structural information for knowledge graph embedding, in: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, pp. 5929–5939.

[27]

Ji S., Pan S., Cambria E., Marttinen P., Yu P.S., A survey on knowledge graphs: Representation, acquisition and applications, 2020, arXiv preprint arXiv:2002.00388.

[28]

Lin M., Chen Q., Yan S., Network in network, 2013, arXiv preprint arXiv:1312.4400.

[29]

C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich, Going deeper with convolutions, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 1–9.

[30]

K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778.

[31]

Z. Sun, S. Vashishth, S. Sanyal, P. Talukdar, Y. Yang, A Re-evaluation of Knowledge Graph Completion Methods, in: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, pp. 5516–5522.

[32]

Jiang X., Wang Q., Wang B., Adaptive convolution for multi-relational learning, in: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Association for Computational Linguistics, Minneapolis, Minnesota, 2019, pp. 978–987.

[33]

Zhang S., Tay Y., Yao L., Liu Q., Quaternion knowledge graph embeddings, in: Advances in Neural Information Processing Systems, 2019, pp. 2735–2745.

[34]

Lacroix T., Usunier N., Obozinski G., Canonical tensor decomposition for knowledge base completion, in: International Conference on Machine Learning, PMLR, 2018, pp. 2863–2872.

[35]

Nguyen D.Q., Vu T., Nguyen T.D., Nguyen D.Q., Phung D., A capsule network-based embedding model for knowledge graph completion and search personalization, in: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Association for Computational Linguistics, Minneapolis, Minnesota, 2019, pp. 2180–2189.

[36]

D. Nathani, J. Chauhan, C. Sharma, M. Kaul, Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs, in: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019, pp. 4710–4723.

[37]

Rossi A., Barbosa D., Firmani D., Matinata A., Merialdo P., Knowledge graph embedding for link prediction: A comparative analysis, ACM Trans. Knowl. Discov. Data (TKDD) 15 (2) (2021) 1–49.

Digital Library

[38]

Kadlec R., Bajgar O., Kleindienst J., Knowledge base completion: Baselines strike back, 2017, arXiv preprint arXiv:1705.10744.

Index Terms

Enhancing the convolution-based knowledge graph embeddings by increasing dimension-wise interactions
1. Computing methodologies
  1. Artificial intelligence
    1. Knowledge representation and reasoning
  2. Machine learning
    1. Machine learning approaches
      1. Learning latent representations
      2. Neural networks
2. Information systems
  1. Information systems applications

Index terms have been assigned to the content through auto-classification.

Recommendations

Knowledge graph embedding in a uniform space

Knowledge graph embedding (KGE) is typically used for link prediction to automatically predict missing links in knowledge graphs. Current KGE models are mainly based on complicated mathematical associations, which are highly expressive but ignore the ...
Knowledge graph embedding model with attention-based high-low level features interaction convolutional network
Abstract
Knowledge graphs are sizeable graph-structured knowledge with both abstract and concrete concepts in the form of entities and relations. Recently, convolutional neural networks have achieved outstanding results for more expressive representations ...
Highlights
- We propose a knowledge graph embedding model with attention-based high-low level feature interaction convolutional network.
- A criss-cross attention mechanism based on consecutive sparsely-connected graphs is introduced to our model.
Learning Embedding for Knowledge Graph Completion with Hypernetwork
Computational Collective Intelligence
Abstract
Link prediction in Knowledge Graph, also called knowledge completion, is a significant problem in graph mining and has many applications for large companies. The more accurate the link prediction results will bring satisfaction, reduce and avoid ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Data & Knowledge Engineering

Data & Knowledge Engineering Volume 146, Issue C

Jul 2023

222 pages

ISSN:0169-023X

Issue’s Table of Contents

Elsevier B.V.

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 01 July 2023

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

0
Total Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 11 Jan 2025

Other Metrics

View Author Metrics

Citations

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents