Abstract
Recent advancements in Large Language Models (LLMs) have primarily focused on enhancing task-specific performances by experimenting with prompt design. Despite the proven effectiveness of Metacognitive Prompting (MP), its application in the field of ontology generation remains an uncharted territory. This study addresses this gap by exploring this prompting technique in supporting the ontology design process, particularly with GPT-4, where this strategy has demonstrated consistent superiority over conventional and more direct prompting methods in recent research. Our methodology, named Ontogenia, employs a gold-standard dataset of ontology competency questions translated into SPARQL-OWL queries. This approach allows us to explore various types and stages of knowledge refinement using MP, while adhering to the eXtreme Design methodology, a well-established protocol in ontology design. Finally, the quality and performance of the resulting ontologies are assessed using both standard ontology quality metrics and evaluation by an ontology expert. This research aims to enrich the discussion on methods of ontology generation driven by LLMs by presenting concrete results on the use of metacognitive prompting and ontology design patterns.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Data and code used for the work is available at this link: https://github.com/dersuchendee/Ontogenia.
- 2.
- 3.
- 4.
- 5.
- 6.
The lack of license information (P41) is also common to all the cases, but this is not an information to be expected from the LLM.
- 7.
Domain and range of those two properties are actually in part inferable because, errors aside, they are meant to be defined in relationship to other properties.
References
Berberette, E., Hutchins, J., Sadovnik, A.: Redefining hallucination in LLMs: towards a psychology-informed framework for mitigating misinformation (2024)
Blomqvist, E., Presutti, V., Daga, E., Gangemi, A.: Experimenting with extreme design. In: Cimiano, P., Pinto, H.S. (eds.) Knowledge Engineering and Management by the Masses, pp. 120–134. Springer, Berlin Heidelberg, Berlin, Heidelberg (2010)
Lai, E.R.: Metacognition: a literature review. Always Learn. Pearson Res. Rep. 24, 1–40 (2011)
Potoniec, J., Wiśniewski, D., Ławrynowicz, A., Keet, C.M.: Dataset of ontology competency questions to SPARQL-OWL queries translations. Data Brief 29, 105098 (2020)
Poveda-Villalón, M., Gómez-Pérez, A., Suárez-Figueroa, M.C.: OOPS! (OntOlogy Pitfall Scanner!): an on-line tool for ontology evaluation. Int. J. Seman. Web Inf. Syst. (IJSWIS) 10(2), 7–34 (2014)
Schraw, G., Moshman, D.: Metacognitive theories. Educ. Psychol. Rev. 7, 351–371 (1995)
Spivack, N., Douglas, S., Crames, M., Connors, T.: Cognition is all you need – the next layer of AI above large language models (2024)
Wang, Y., Zhao, Y.: Metacognitive prompting improves understanding in large language models. arXiv preprint arXiv:2308.05342 (2023)
Zhou, Y., Liu, Z., Jin, J., Nie, J.Y., Dou, Z.: Metacognitive retrieval-augmented large language models. arXiv preprint arXiv:2402.11626 (February 2024), https://arxiv.org/abs/2402.11626
Acknowledgements
This work was supported by (i) by FOSSR (Fostering Open Science in Social Science Research), funded by the European Union - NextGenerationEU under NRRP Grant agreement n. MUR IR0000008; and (ii) the European Union’s Horizon Europe research and innovation programme within the context of the project HACID (Hybrid Human Artificial Collective Intelligence in Open-Ended Domains, grant agreement No 101070588).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Disclaimer
The content of this article reflects only the authors’ view. The European Commission and the Italian Ministry of University and Research are not responsible for any use that may be made of the information it contains.
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Lippolis, A.S., Ceriani, M., Zuppiroli, S., Nuzzolese, A.G. (2025). Ontogenia: Ontology Generation with Metacognitive Prompting in Large Language Models. In: Meroño Peñuela, A., et al. The Semantic Web: ESWC 2024 Satellite Events. ESWC 2024. Lecture Notes in Computer Science, vol 15344. Springer, Cham. https://doi.org/10.1007/978-3-031-78952-6_38
Download citation
DOI: https://doi.org/10.1007/978-3-031-78952-6_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-78951-9
Online ISBN: 978-3-031-78952-6
eBook Packages: Computer ScienceComputer Science (R0)