Cited By
View all- Huang CYu FWan ZLi FJi HLi Y(2024)Knowledge graph confidence-aware embedding for recommendationNeural Networks10.1016/j.neunet.2024.106601180(106601)Online publication date: Dec-2024
AspectMMKG: A Multi-modal Knowledge Graph with Aspect-aware Entities
Pages 3361 - 3370
Abstract
Multi-modal knowledge graphs (MMKGs) combine different modal data (e.g., text and image) for a comprehensive understanding of entities. Despite the recent progress of large-scale MMKGs, existing MMKGs neglect the multi-aspect nature of entities, limiting the ability to comprehend entities from various perspectives.In this paper, we construct AspectMMKG, the first MMKG with aspect-related images by matching images to different entity aspects. Specifically, we collect aspect-related images from a knowledge base, and further extract aspect-related sentences from the knowledge base as queries to retrieve a large number of aspect-related images via an online image search engine. Finally, AspectMMKG contains 2,380 entities, 18,139 entity aspects, and 645,383 aspect-related images. We demonstrate the usability of AspectMMKG in entity aspect linking (EAL) downstream task and show that previous EAL models achieve a new state-of-the-art performance with the help of AspectMMKG.To facilitate the research on aspect-related MMKG, we further propose an aspect-related image retrieval (AIR) model, that aims to correct and expand aspect-related images in AspectMMKG.We train an AIR model to learn the relationship between entity image and entity aspect-related images by incorporating entity image, aspect, and aspect image information. Experimental results indicate that the AIR model could retrieve suitable images for a given entity w.r.t different aspects.
References
[1]
Houda Alberts, Teresa Huang, Yash Deshpande, Yibo Liu, Kyunghyun Cho, Clara Vania, and Iacer Calixto. 2020. Visualsem: a high-quality knowledge graph for vision and language. arXiv preprint arXiv:2008.09150 (2020).
[2]
Sebastian Arnold, Rudolf Schneider, Philippe Cudré-Mauroux, Felix A Gers, and Alexander Löser. 2019. Sector: A neural model for coherent topic segmentation and classification. Transactions of the Association for Computational Linguistics 7 (2019), 169--184.
[3]
Sebastian Arnold, Betty van Aken, Paul Grundmann, Felix A Gers, and Alexander Löser. 2020. Learning contextualized document representations for healthcare answer retrieval. In Proceedings of The Web Conference 2020. 1332--1343.
[4]
Niranjan Balasubramanian and Silviu Cucerzan. 2009. Automatic generation of topic pages using query-based aspect models. In Proceedings of the 18th ACM conference on Information and knowledge management. 2049--2052.
[5]
Siddhartha Banerjee and Prasenjit Mitra. 2015. Wikikreator: Improving wikipedia stubs automatically. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 867--877.
[6]
Sebastián Ferrada, Benjamin Bustos, and Aidan Hogan. 2017. IMGpedia: a linked dataset with content-based analysis of Wikimedia images. In The Semantic Web-- ISWC 2017: 16th International Semantic Web Conference, Vienna, Austria, October 21--25, 2017, Proceedings, Part II 16. Springer, 84--93.
[7]
Besnik Fetahu, Katja Markert, and Avishek Anand. 2015. Automated news suggestions for populating wikipedia entity pages. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. 323--332.
[8]
Joseph L Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological bulletin 76, 5 (1971), 378.
[9]
Peng Li, Yinglin Wang, Wei Gao, and Jing Jiang. 2011. Generating aspect-oriented multi-document summarization with event-aspect model. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing. 1137--1146.
[10]
Yuqing Li, Yuxin Zhang, Bin Wu, Ji-Rong Wen, Ruihua Song, and Ting Bai. 2022. A Multi-Modal Knowledge Graph for Classical Chinese Poetry. In Findings of the Association for Computational Linguistics: EMNLP 2022. Association for Computational Linguistics, Abu Dhabi, United Arab Emirates, 2318--2326. https://doi.org/10.18653/v1/2022.findings-emnlp.171
[11]
Ye Liu, Hui Li, Alberto Garcia-Duran, Mathias Niepert, Daniel Onoro-Rubio, and David S Rosenblum. 2019. MMKG: multi-modal knowledge graphs. In The Semantic Web: 16th International Conference, ESWC 2019, Portoro?, Slovenia, June 2--6, 2019, Proceedings 16. Springer, 459--474.
[12]
Federico Nanni, Simone Paolo Ponzetto, and Laura Dietz. 2018. Entity-aspect linking: providing fine-grained semantics of entities in context. In Proceedings of the 18th ACM/IEEE on Joint Conference on Digital Libraries. 49--58.
[13]
Daniel Oñoro-Rubio, Mathias Niepert, Alberto García-Durán, Roberto González, and Roberto J López-Sastre. 2017. Answering visual-relational queries in web- extracted knowledge graphs. arXiv preprint arXiv:1709.02314 (2017).
[14]
Patrick Pantel, Thomas Lin, and Michael Gamon. 2012. Mining entity types from query logs via user intent modeling. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 563--571.
[15]
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. arXiv:2103.00020 [cs.CV]
[16]
Jordan S. Ramsdell and Laura Dietz. 2020. A Large Test Collection for Entity Aspect Linking. Proceedings of the 29th ACM International Conference on Information & Knowledge Management (2020).
[17]
Ridho Reinanda, Edgar Meij, and Maarten de Rijke. 2015. Mining, ranking and recommending entity aspects. In Proceedings of the 38th international acm sigir conference on research and development in information retrieval. 263--272.
[18]
Christina Sauper and Regina Barzilay. 2009. Automatically generating wikipedia articles: A structure-aware approach. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP. 208--216.
[19]
Haoxiang Shi, Rongsheng Zhang, Jiaan Wang, Cen Wang, Yinhe Zheng, and Tetsuya Sakai. 2022. LayerConnect: Hypernetwork-Assisted Inter-Layer Connector to Enhance Parameter Efficiency. In Proceedings of the 29th International Conference on Computational Linguistics. 3120--3126.
[20]
Bilyana Taneva and Gerhard Weikum. 2013. Gem-based entity-knowledge maintenance. In Proceedings of the 22nd ACM international conference on Information & Knowledge Management. 149--158.
[21]
Jiaan Wang, Zhixu Li, Tingyi Zhang, Duo Zheng, Jianfeng Qu, An Liu, Lei Zhao, and Zhigang Chen. 2022. Knowledge enhanced sports game summarization. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining. 1045--1053.
[22]
Jiaan Wang, Yunlong Liang, Fandong Meng, Haoxiang Shi, Zhixu Li, Jinan Xu, Jianfeng Qu, and Jie Zhou. 2023. Is chatgpt a good nlg evaluator? a preliminary study. arXiv preprint arXiv:2303.04048 (2023).
[23]
Jiaan Wang, Yunlong Liang, Fandong Meng, Beiqi Zou, Zhixu Li, Jianfeng Qu, and Jie Zhou. 2023. Zero-Shot cross-lingual summarization via large language models.
[24]
Jiaan Wang, Fandong Meng, Ziyao Lu, Duo Zheng, Zhixu Li, Jianfeng Qu, and Jie Zhou. 2022. ClidSum: A Benchmark Dataset for Cross-Lingual Dialogue Summarization. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 7716--7729.
[25]
Jiaan Wang, Fandong Meng, Duo Zheng, Yunlong Liang, Zhixu Li, Jianfeng Qu, and Jie Zhou. 2022. A survey on cross-lingual summarization. Transactions of the Association for Computational Linguistics 10 (2022), 1304--1323.
[26]
Jiaan Wang, Fandong Meng, Duo Zheng, Yunlong Liang, Zhixu Li, Jianfeng Qu, and Jie Zhou. 2023. Towards Unifying Multi-Lingual and Cross-Lingual Summarization. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Toronto, Canada, 15127--15143. https://doi.org/10.18653/v1/2023.acl-long.843
[27]
Meng Wang, Guilin Qi, HaoFen Wang, and Qiushuo Zheng. 2020. Richpedia: a comprehensive multi-modal knowledge graph. In Semantic Technology: 9th Joint International Conference, JIST 2019, Hangzhou, China, November 25--27, 2019, Proceedings 9. Springer, 130--145.
[28]
Xiaoxin Yin and Sarthak Shah. 2010. Building taxonomy of web search intents for name entity queries. In Proceedings of the 19th international conference on World wide web. 1001--1010.
[29]
Xiaoming Zhang, Xin Liu, Xin Li, and Dongyu Pan. 2017. MMKG: An approach to generate metallic materials knowledge graph based on DBpedia and Wikipedia. Computer Physics Communications 211 (2017), 98--112.
[30]
Xiaoming Zhang, Xiaoling Sun, Chunjie Xie, and Bing Lun. 2019. From vision to content: Construction of domain-specific multi-modal knowledge graph. IEEE Access 7 (2019), 108278--108294.
[31]
Wentian Zhao, Yao Hu, Heda Wang, Xinxiao Wu, and Jiebo Luo. 2021. Boosting entity-aware image captioning with multi-modal knowledge graph. arXiv preprint arXiv:2107.11970 (2021).
[32]
Xiangru Zhu, Zhixu Li, Xiaodan Wang, Xueyao Jiang, Penglei Sun, Xuwu Wang, Yanghua Xiao, and Nicholas Jing Yuan. 2022. Multi-modal knowledge graph construction and application: A survey. IEEE Transactions on Knowledge and Data Engineering (2022).
Index Terms
- AspectMMKG: A Multi-modal Knowledge Graph with Aspect-aware Entities
Recommendations
Hyper-node Relational Graph Attention Network for Multi-modal Knowledge Graph Completion
Knowledge graphs often suffer from incompleteness, and knowledge graph completion (KGC) aims at inferring the missing triplets through knowledge graph embedding from known factual triplets. However, most existing knowledge graph embedding methods only use ...
MMpedia: A Large-Scale Multi-modal Knowledge Graph
The Semantic Web – ISWC 2023AbstractKnowledge graphs serve as crucial resources for various applications. However, most existing knowledge graphs present symbolic knowledge in the form of natural language, lacking other modal information, e.g., images. Previous multi-modal knowledge ...
A Probabilistic Inference Based Approach for Querying Associative Entities in Knowledge Graph
Web and Big DataAbstractQuerying associative entities is to provide top ranked entities in knowledge graph (KG). Many entities are not linked explicitly in KG but actually associated when incorporating outside user-generated data, which could enrich entity associations ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
October 2023
5508 pages
ISBN:9798400701245
DOI:10.1145/3583780
- General Chairs:
- Ingo Frommholz,
- Frank Hopfgartner,
- Mark Lee,
- Michael Oakes,
- Program Chairs:
- Mounia Lalmas,
- Min Zhang,
- Rodrygo Santos
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 21 October 2023
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- Shanghai Science and Technology Innovation Action Plan
- Shanghai Municipal Science and Technology Major Project
- National Key Research and Development Project
- National Natural Science Foundation of China
- Science and Technology Commission of Shanghai Municipality Grant
Conference
CIKM '23: The 32nd ACM International Conference on Information and Knowledge Management
October 21 - 25, 2023
Birmingham, United Kingdom
Acceptance Rates
Overall Acceptance Rate 1,861 of 8,427 submissions, 22%
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 180Total Downloads
- Downloads (Last 12 months)154
- Downloads (Last 6 weeks)18
Reflects downloads up to 21 Nov 2024
Other Metrics
Citations
Cited By
View all- Huang CYu FWan ZLi FJi HLi Y(2024)Knowledge graph confidence-aware embedding for recommendationNeural Networks10.1016/j.neunet.2024.106601180(106601)Online publication date: Dec-2024
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in