Article

On how to perform a gold standard based evaluation of ontology learning

Authors:

Klaas Dellschaft,

Steffen StaabAuthors Info & Claims

ISWC'06: Proceedings of the 5th international conference on The Semantic Web

Pages 228 - 241

https://doi.org/10.1007/11926078_17

Published: 05 November 2006 Publication History

Abstract

In recent years several measures for the gold standard based evaluation of ontology learning were proposed. They can be distinguished by the layers of an ontology (e.g. lexical term layer and concept hierarchy) they evaluate. Judging those measures with a list of criteria we show that there exist some measures sufficient for evaluating the lexical term layer. However, existing measures for the evaluation of concept hierarchies fail to meet basic criteria. This paper presents a new taxonomic measure which overcomes the problems of current approaches.

References

[1]

Cimiano, P., Pivk, A., Schmidt-Thieme, L., Staab, S.: Learning taxonomic relations from heterogenous sources of evidence. In: Ontology Learning from Text: Methods, Applications, Evaluation. IOS Press (2005).

[2]

Spyns, P., Reinberger, M.L.: Lexically evaluating ontology triples generated automatically from texts. In: Proc. of the second European Conference on the Semantic Web. (2005).

Digital Library

[3]

Sabou, M., Wroe, C., Goble, C., Stuckenschmidt, H.: Learning domain ontologies for semantic web service descriptions. Journal of Web Semantics 3(4) (2005).

Digital Library

[4]

Sabou, M., Wroe, C., Goble, C., Mishne, G.: Learning domain ontologies for web service descriptions: an experiment in bioinformatics. In: Proc. of WWW05. (2005).

Digital Library

[5]

Reinberger, M.L., Spyns, P.: Unsupervised text mining for the learning of dogma-inspired ontologies. (In: Ontology Learning from Text: Methods, Applications and Evaluation).

[6]

Maedche, A.: Ontology Learning for the Semantic Web. Kluwer, Boston (2002).

Digital Library

[7]

Maedche, A., Staab, S.: Measuring similarity between ontologies. In: Proc. of the European Conference on Knowledge Acquisition and Management (EKAW-2002). (2002).

Digital Library

[8]

Cimiano, P., Hotho, A., Staab, S.: Learning concept hierarchies from text corpora using formal concept analysis. JAIR - Journal of AI Research 24 (2005) 305-339.

Digital Library

[9]

Maynard, D., Peters, W., Li, Y.: Metrics for evaluation of ontology-based information extraction. In: Proc. of the EON 2006 Workshop. (2006).

[10]

Hahn, U., Schnattinger, K.: Towards text knowledge engineering. In: Proc. of the 15th National Conference on Artificial Intelligence (AAAI-98). (1998).

Digital Library

[11]

Brank, J., Mladenic, D., Grobelnik, M.: Gold standard based ontology evaluation using instance assignment. In: Proc. of the EON 2006 Workshop. (2006).

[12]

Hartmann, J., Spyns, P., Maynard, D., Cuel, R., Carmen Suarez de Figueroa, M., Sure, Y.: Methods for ontology evaluation. Deliverable D1.2.3, Knowledge Web (2004).

[13]

van Rijsbergen, C.: Information Retrieval. Butterworths, London (1979).

Digital Library

[14]

Dellschaft, K.: Measuring the similiarity of concept hierarchies and its influence on the evaluation of learning procedures. Diploma thesis, Universität Koblenz-Landau (2005) http://www.uni-koblenz.de/FB4/Institutes/IFI/AGStaab/Theses/2005/DADellschaft.pdf.

[15]

Hearst, M.: Automatic aquisition of hyponyms from large text corpora. In: Proc. of the 14th International Conference on Computational Linguistics. (1992).

Digital Library

[16]

Staab, S., et al.: Getess - searching the web exploiting german texts. In: Proc. of the 3rd Workshop on Cooperative Information Agents. (1999).

Digital Library

Cited By

Alfonso-Hermelo DLanglais PBourg L(2019)Automatically Learning a Human-Resource Ontology from Professional Social-Network DataAdvances in Artificial Intelligence10.1007/978-3-030-18305-9_11(132-145)Online publication date: 28-May-2019
https://dl.acm.org/doi/10.1007/978-3-030-18305-9_11
Sathiya BGeetha T(2018)Automatic Ontology Learning from Multiple Knowledge Sources of TextInternational Journal of Intelligent Information Technologies10.4018/IJIIT.201804010114:2(1-21)Online publication date: 1-Apr-2018
https://dl.acm.org/doi/10.4018/IJIIT.2018040101
Zong NKim HNam S(2017)Constructing faceted taxonomy for heterogeneous entities based on object properties in linked dataData & Knowledge Engineering10.1016/j.datak.2017.09.006112:C(79-93)Online publication date: 1-Nov-2017
https://dl.acm.org/doi/10.1016/j.datak.2017.09.006
Show More Cited By

On how to perform a gold standard based evaluation of ontology learning
1. Computing methodologies

Recommendations

Gold Standard Evaluation of Ontology Learning Methods through Ontology Transformation and Alignment

This paper presents a method along with a set of measures for evaluating learned ontologies against gold ontologies. The proposed method transforms the ontology concepts and their properties into a vector space representation to avoid the common string ...
Gold standard based evaluation of ontology learning techniques
SAC '16: Proceedings of the 31st Annual ACM Symposium on Applied Computing

A growing attention has been paid to the ontology learning domain. This is due to its importance for overcoming the limits of manual ontology building. Thus, ontology evaluation becomes crucial and very much-needed in order to select the best performing ...
Ontology Evaluation and Multilingualism
CompSysTech '20: Proceedings of the 21st International Conference on Computer Systems and Technologies

Many ontologies have been developed recently, and evaluation is important for researchers or users when searching ontologies needed in their work. Ontology evaluation also is important phase of ontology development, refinement or evolution process. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

ISWC'06: Proceedings of the 5th international conference on The Semantic Web

November 2006

1001 pages

ISBN:3540490299

Editors:
Isabel Cruz
Department of Computer Science, University of Illinois at Chicago, 851 South Morgan Street (M/C 152), Chicago, IL
,
Stefan Decker
Digital Enterprise Research Institute, National University of Ireland, Galway, IDA Business Park, Lower Dangan, 851 South Morgan Street (M/C 152), Galway, IL, Ireland
,
Dean Allemang
Digital Enterprise Research Institute, TopQuadrant, 851 South Morgan Street (M/C 152), Galway, VA, Ireland
,
Chris Preist
Digital Enterprise Research Institute, HP Laboratories, 851 South Morgan Street (M/C 152), Bristol, VA, UK
,
Daniel Schwabe
Departamento de Informática - Pontifícia, Universidade Católica do Rio de Janeiro,, (PUC Rio) - Caixa Postal 38.097, Rio de Janeiro, RJ, Brazil

Sponsors

Ontotext
Semantics Utilised for Process Management within and between Enterprises: Semantics Utilised for Process Management within and between Enterprises
Aduna: Aduna
Vulcan
Data, Information, and Process Integration with Semantic Web Services: Data, Information, and Process Integration with Semantic Web Services

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 05 November 2006

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

45
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Alfonso-Hermelo DLanglais PBourg L(2019)Automatically Learning a Human-Resource Ontology from Professional Social-Network DataAdvances in Artificial Intelligence10.1007/978-3-030-18305-9_11(132-145)Online publication date: 28-May-2019
https://dl.acm.org/doi/10.1007/978-3-030-18305-9_11
Sathiya BGeetha T(2018)Automatic Ontology Learning from Multiple Knowledge Sources of TextInternational Journal of Intelligent Information Technologies10.4018/IJIIT.201804010114:2(1-21)Online publication date: 1-Apr-2018
https://dl.acm.org/doi/10.4018/IJIIT.2018040101
Zong NKim HNam S(2017)Constructing faceted taxonomy for heterogeneous entities based on object properties in linked dataData & Knowledge Engineering10.1016/j.datak.2017.09.006112:C(79-93)Online publication date: 1-Nov-2017
https://dl.acm.org/doi/10.1016/j.datak.2017.09.006
(2016)Refactoring ontologies using design patterns and relational concepts analysis to integrate viewsInternational Journal of Metadata, Semantics and Ontologies10.1504/IJMSO.2016.1000426011:4(243-263)Online publication date: 1-Jan-2016
https://dl.acm.org/doi/10.1504/IJMSO.2016.10004260
Dias SDolianiti FHadjileontiadou SDiniz JHadjileontiadis L(2016)FISCMAPProceedings of the 7th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion10.1145/3019943.3019986(293-300)Online publication date: 1-Dec-2016
https://dl.acm.org/doi/10.1145/3019943.3019986
Sathiya BGeetha TAkila VSivakumar NSaruladha KZayaraz GIlavarasan E(2016)Semantic Querying based Concept Hierarchy Construction for Ontology LearningProceedings of the International Conference on Informatics and Analytics10.1145/2980258.2980307(1-6)Online publication date: 25-Aug-2016
https://dl.acm.org/doi/10.1145/2980258.2980307
Sfar HChaibi ABouzeghoub AGhezala HOssowski S(2016)Gold standard based evaluation of ontology learning techniquesProceedings of the 31st Annual ACM Symposium on Applied Computing10.1145/2851613.2851843(339-346)Online publication date: 4-Apr-2016
https://dl.acm.org/doi/10.1145/2851613.2851843
Oramas SSordo MEspinosa-Anke LGangemi ALeonardi SPanconesi A(2015)A Rule-Based Approach to Extracting Relations from Music TidbitsProceedings of the 24th International Conference on World Wide Web10.1145/2740908.2741709(661-666)Online publication date: 18-May-2015
https://dl.acm.org/doi/10.1145/2740908.2741709
Silva FGirardi R(2014)An Approach to Join Ontologies and Their Reuse in the Construction of Application OntologiesProceedings of the 2014 IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT) - Volume 0110.1109/WI-IAT.2014.66(424-431)Online publication date: 11-Aug-2014
https://dl.acm.org/doi/10.1109/WI-IAT.2014.66
Albukhitan SHelmy T(2014)Multi-agent Based System for Multilingual Ontologies MaintenanceProceedings of the 2014 IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT) - Volume 0110.1109/WI-IAT.2014.65(419-423)Online publication date: 11-Aug-2014
https://dl.acm.org/doi/10.1109/WI-IAT.2014.65
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Table of Contents