Abstract
As a key part of knowledge graph embedding (KGE), negative sampling can mine hard negative examples required for model training to improve the accuracy and effectiveness of the KGE models. However, static negative sampling can’t adapt to the dynamic distribution of hard negative examples, causing the model to miss some important negative examples during the model training. Dynamic negative sampling, although it can adapt to the changes in the data, is unable to mine and select enough hard negative examples. So most existing negative samplings affect the model’s comprehension and representation of the data. In this paper, we propose a negative example sampling method by mixup (NESI), which can generate a large number of synthetic examples at the feature-level to solve the problems of lack of hard negative examples and gradient vanishing. Our method utilizes two data mixing strategies and four evaluation functions to rapidly expand the size of the negative candidates, and effectively improve the quality of them. Therefore, NESI can efficiently select enough high-quality hard negative examples. Compared with the existing negative samplings, NESI improves the model performance on MRR by 10.58% and greatly accelerates the convergence speed of the KGE models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Adler, J., Lunz, S.: Banach wasserstein GAN. Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018 pp. 6755–6764 (2018)
Ahrabian, K., Feizi, A., Salehi, Y., Hamilton, W.L., Bose, A.J.: Structure aware negative sampling in knowledge graphs. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. pp. 6093–6101. Association for Computational Linguistics (2020)
Auer, S., Bizer, C., Kobilarov, G., Lehmann, J., Cyganiak, R., Ives, Z.: Dbpedia: A nucleus for a web of open data. In: The Semantic Web, 6th International Semantic Web Conference, 2nd Asian Semantic Web Conference. vol. 4825, pp. 722–735. Springer (2007)
Bollacker, K., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: A collaboratively created graph database for structuring human knowledge. In: Proceedings of the 2008 ACM SIGMOD international conference on Management of data. pp. 1247–1250. ACM (2008)
Bordes, A., Glorot, X., Weston, J., Bengio, Y.: A semantic matching energy function for learning with multi-relational data. Mach. Learn. 94(2), 233–259 (2014)
Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. Advances in neural information processing systems 26: Annual Conference on Neural Information Processing Systems 2013 pp. 2787–2795 (2013)
Cai, L., Wang, W.Y.: Kbgan: Adversarial learning for knowledge graph embeddings. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). pp. 1470–1480. Association for Computational Linguistics (2018)
Che, F., Yang, G., Shao, P., Zhang, D., Tao, J.: Mixkg: Mixing for harder negative samples in knowledge graph. CoRR abs/2202.09606 (2022), https://arxiv.org/abs/2202.09606
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014. pp. 2672–2680 (2014)
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial networks. Commun. ACM 63(11), 139–144 (2020)
Gu, Z., Wang, L., Chen, X., Tang, Y., Wang, X., Du, X., Guizani,M., Tian, Z.: Epidemic risk assessment by a novel communication station based method. IEEE Trans. Netw. Sci. Eng. 9(1), 332–344 (2022). https://doi.org/10.1109/TNSE.2021.3058762, https://doi.org/10.1109/TNSE.2021.3058762
Guo, Q., Zhuang, F., Qin, C., Zhu, H., Xie, X., Xiong, H., He, Q.: A survey on knowledge graph-based recommender systems. IEEE Trans. Knowl. Data Eng. 34(8), 3549–3568 (2022)
Hao, Y., Zhang, Y., Liu, K., He, S., Liu, Z., Wu, H., Zhao, J.: An end-to-end model for question answering over knowledge base with cross-attention combining global knowledge. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 221–231. Association for Computational Linguistics (2017)
Islam, M.K., Aridhi, S., Smail-Tabbone, M.: Negative sampling and rule mining for explainable link prediction in knowledge graphs. Knowl.-Based Syst. 250, 109083 (2022)
Kalantidis, Y., Sariyildiz, M.B., Pion, N., Weinzaepfel, P., Larlus, D.: Hard negative mixing for contrastive learning. Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020 33, 21798–21809 (2020)
Li, L., Zhang, X., Ma, Y., Gao, C., Wang, J., Yu, Y., Yuan, Z., Ma, Q.: A knowledge graph completion model based on contrastive learning and relation enhancement method. Knowl.-Based Syst. 256, 109889 (2022)
Nickel, M., Tresp, V., Kriegel, H.P.: A three-way model for collective learning on multi-relational data. In: Proceedings of the 28th International Conference on Machine Learning. pp. 809–816. Omnipress (2011)
Nickel, M., Murphy, K., Tresp, V., Gabrilovich, E.: A review of relational machine learning for knowledge graphs. Proc. IEEE 104(1), 11–33 (2016)
Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., Lerer, A.: Automatic differentiation in pytorch. In: Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017 (2017), https://openreview.net/forum?id=BJJsrmfCZ
Shen, Z., Liu, Z., Liu, Z., Savvides, M., Darrell, T., Xing, E.: Un-mix: Rethinking image mixtures for unsupervised visual representation learning. In: Proceedings of the 36th AAAI Conference on Artificial Intelligence. pp. 2216–2224. AAAI Press (2022)
Sun, Z., Deng, Z.H., Nie, J.Y., Tang, J.: Rotate: Knowledge graph embedding by relational rotation in complex space. In: Proceedings of the 7th International Conference on Learning Representations (2019), https://openreview.net/forum?id=HkgEQnRqYQ
Trouillon, T., Dance, C., Gaussier, E., Welbl, J., Riedel, S., Bouchard, G.: Knowledge graph completion via complex tensor factorization. J. Mach. Learn. Res. 18(130), 1–38 (2017)
Verma, V., Lamb, A., Beckham, C., Najafi, A., Mitliagkas, I., Lopez-Paz, D., Bengio, Y.: Manifold mixup: Better representations by interpolating hidden states. In: Proceedings of the 36th International Conference on Machine Learning. vol. 97, pp. 6438–6447. PMLR (2019)
Wang, Q., Mao, Z., Wang, B., Guo, L.: Knowledge graph embedding: A survey of approaches and applications. IEEE Trans. Knowl. Data Eng. 29(12), 2724–2743 (2017)
Wang, Z., Zhang, J., Feng, J., Chen, Z.: Knowledge graph embedding by translating on hyperplanes. In: Proceedings of the 28th AAAI conference on artificial intelligence. pp. 1112–1119. AAAI Press (2014)
Xiong, C., Power, R., Callan, J.: Explicit semantic ranking for academic search via knowledge graph embedding. In: Proceedings of the 26th international conference on world wide web. pp. 1271–1279. ACM (2017)
Yang, B., Yih, S.W.t., He, X., Gao, J., Deng, L.: Embedding entities and relations for learning and inference in knowledge bases. In: Proceedings of the 3rd International Conference on Learning Representations (2015), http://arxiv.org/abs/1412.6575
Yao, X., Van Durme, B.: Information extraction over structured data: Question answering with freebase. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 956–966. The Association for Computer Linguistics (2014)
Zhang, H., Cisse, M., Dauphin, Y.N., Lopez-Paz, D.: mixup: Beyond empirical risk minimization. In: Proceedings of the 6th International Conference on Learning Representations (2018), https://openreview.net/forum?id=r1Ddp1-Rb
Zhang, Y., Yao, Q., Shao, Y., Chen, L.: Nscaching: Simple and efficient negative sampling for knowledge graph embedding. In: Proceedings of the 35th IEEE International Conference on Data Engineering. pp. 614–625. IEEE (2019)
Acknowledgments
This work is supported in part by the Major Key Project of PCL (PCL2022A03), and National Natural Science Foundation of China (Grant No. 62372137).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Xie, Y., Wang, H., Wang, L., Luo, L., Li, J., Gu, Z. (2024). Reinforced Negative Sampling for Knowledge Graph Embedding. In: Onizuka, M., et al. Database Systems for Advanced Applications. DASFAA 2024. Lecture Notes in Computer Science, vol 14853. Springer, Singapore. https://doi.org/10.1007/978-981-97-5562-2_23
Download citation
DOI: https://doi.org/10.1007/978-981-97-5562-2_23
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-5561-5
Online ISBN: 978-981-97-5562-2
eBook Packages: Computer ScienceComputer Science (R0)