Nothing Special   »   [go: up one dir, main page]
Skip to main content
SpringerLink
Log in

Towards metrics-driven ontology engineering

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

The software engineering field is continuously making an effort to improve the effectiveness of the software development process. This improvement is performed by developing quantitative measures that can be used to enhance the quality of software products and to more accurately describe, better understand and manage the software development life cycle. Even if the ontology engineering field is constantly adopting practices from software engineering, it has not yet reached a state in which metrics are an integral part of ontology engineering processes and support making evidence-based decisions over the process and its outputs. Up to now, ontology metrics are mainly focused on the ontology implementation and do not take into account the development process or other artefacts that can help assessing the quality of the ontology, e.g. its requirements. This work envisions the need for a metrics-driven ontology engineering process and, as a first step, presents a set of metrics for ontology engineering which are obtained from artefacts generated during the ontology development process and from the process itself. The approach is validated by measuring the ontology engineering process carried out in a research project and by showing how the proposed metrics can be used to improve the efficiency of the process by making predictions, such as the effort needed to implement an ontology, or assessments, such as the coverage of the ontology according to its requirements.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. https://protege.stanford.edu/.

  2. https://webprotege.stanford.edu/.

  3. The authors of the analysed paper refer to the knowledge base as the set of TBox and Abox.

  4. https://github.com/.

  5. https://www.w3.org/TR/rdf-sparql-query/.

  6. http://vicinity2020.eu/vicinity/.

  7. http://iot.linkeddata.es/def/core/.

  8. http://iot.linkeddata.es/def/wot/.

  9. http://iot.linkeddata.es/def/wot-mappings/.

  10. http://iot.linkeddata.es/def/adapters/.

  11. http://iot.linkeddata.es/def/datatypes/.

  12. https://www.w3.org/WoT/WG/.

  13. http://vicinity.iot.linkeddata.es.

  14. The link to all the Github repositories are indicated in the VICINITY ontology portal: http://vicinity.iot.linkeddata.es/ which due to its version control allows the ontology engineers to be aware of the evolution of the artefacts during the development iterations. Test suites are also stored in the GitHub repository of its associated ontology.

References

  1. Albrecht AJ (1979) Measuring application development productivity. In: Proceedings of the joint SHARE/GUIDE/IBM application development symposium, pp 83–92

  2. Baader F, Horrocks I, Sattler U (2008) Description logics. Found Artif Intell 3:135–179

    Article  Google Scholar 

  3. Blomqvist E, Sepour AS, Presutti V (2012) Ontology testing-methodology and tool. In: Proceedings of the 18th international conference on knowledge engineering and knowledge management, Galway City, Ireland, October 8–12. Springer, Berlin, pp 216–226

  4. Boehm B, Clark B, Horowitz E, Westland C, Madachy R, Selby R (1995) Cost models for future software life cycle processes: COCOMO 2.0. Ann Softw Eng 1(1):57–94

    Article  Google Scholar 

  5. Brickley D, Guha RV, McBride B (2014) RDF Schema 1.1. W3C Recommendation 25 February 2014. Available at https://www.w3.org/TR/rdf-schema/

  6. Costello RJ, Liu DB (1995) Metrics for requirements engineering. J Syst Softw 29(1):39–63

    Article  Google Scholar 

  7. Davis A, Overmyer S, Jordan K, Caruso J, Dandashi F, Dinh A, Kincaid G, Ledeboer G, Reynolds P, Sitaram P, Ta A, Theofanos M (1993) Identifying and measuring quality in a software requirements specification. In: Proceedings of the 1st international software metrics symposium, Baltimore, MD, USA, May 21–22. IEEE, pp 141–152

  8. De Leenheer P, Debruyne C (2008) DOGMA-MESS: a tool for fact-oriented collaborative ontology evolution. In: Proceedings of the 2008 international conference on On the move to meaningful internet systems: OTM 2008 Workshops, Monterrey, Mexico, November 9–14. Springer, Berlin, pp 797–806

  9. De Moor A, De Leenheer P, Meersman R (2006) DOGMA-MESS: a meaning evolution support system for interorganizational ontology engineering. In: Proceedings of the 14th international conference on conceptual structures, Aalborg, Denmark, July 16–21. Springer, Berlin, pp 189–202

  10. De Nicola A, Missikoff M, Navigli R (2005) A proposal for a unified process for ontology building: Upon. In: Proceedings of the 16th international conference on database and expert systems applications, Copenhagen, Denmark, August 22–26. Springer, Berlin, pp 655–664

  11. Debruyne C, Tran TK, Meersman R (2013) Grounding ontologies with social processes and natural language. J Data Semant 2(2–3):89–118

    Article  Google Scholar 

  12. DeMarco T (1979) Structured analysis and system specification. Yourdon Press, Berlin

  13. Duque-Ramos A, Fernández-Breis JT, Iniesta M, Dumontier M, Aranguren ME, Schulz S, Aussenac-Gilles N, Stevens R (2013) Evaluation of the OQuaRE framework for ontology quality. Expert Syst Appl 40(7):2696–2703

    Article  Google Scholar 

  14. Fenton N, Bieman J (1997) Software metrics: a rigorous and practical approach. PWS Publishing Company, Boston

    MATH  Google Scholar 

  15. Fenton NE, Neil M (2000) Software metrics: roadmap. In: Proceedings of the conference on the future of software engineering, Limerick, Ireland, June 04–11. ACM, pp 357–370

  16. Fernández-López M, Gómez-Pérez A (2002) The integration of OntoClean in WebODE. In: Proceedings of the OntoWeb-SIG3 workshop at the 13th international conference on knowledge engineering and knowledge management, Siguenza, Spain, 30th September. CEUR-WS.org, CEUR Workshop Proceedings, vol 62, pp 38–52

  17. Fernández-López M, Gómez-Pérez A, Juristo N (1997) Methontology: from ontological art towards ontological engineering. In: Proceedings of the ontological engineering AAAI97 spring symposium series. Stanford University, EEUU, March 24–26. AAAI Press, pp 33–40

  18. Gangemi A, Presutti V (2009) Ontology design patterns. Handbook on ontologies. Springer, Berlin, pp 221–243

  19. Gangemi A, Catenacci C, Ciaramita M, Lehmann J (2006) Modelling ontology evaluation and validation. In: Proceedings of the 3rd European Semantic Web Conference, Budva, Montenegro, June 11–14. Springer, Berlin, pp 140–154

  20. García-Ramos S, Otero A, Fernández-López M (2009) OntologyTest: A tool to evaluate ontologies through tests defined by the user. In: Proceedings of the 10th international work-conference on artificial neural networks on artificial neural networks, Salamanca, Spain, June 10–12. Springer, Berlin, pp 91–98

  21. Gunning R (1952) The technique of clear writing. McGraw-Hill, New York

    Google Scholar 

  22. Hitzler P, Krötzsch M, Parsia B, Patel-Schneider PF, Rudolph S (2009) OWL 2 Web Ontology Language Primer (Second Edition) W3C Recommendation 11 December 2012. Available at https://www.w3.org/TR/owl2-primer/

  23. Iqbal S, Naeem M, Khan A (2012) Yet another set of requirement metrics for software projects. Int J Softw Eng Its Appl 6(1):19–28

    Google Scholar 

  24. Kan SH (2002) Metrics and models in software quality engineering. Addison-Wesley, London

    MATH  Google Scholar 

  25. Kang YB, Li YF, Krishnaswamy S (2012) Predicting reasoning performance using ontology metrics. In: Proceedings of the 11th international semantic web conference, Boston, MA, USA, November 11–15, Springer, Berlin, pp 198–214

  26. Keet CM, Ławrynowicz A (2016) Test-driven development of ontologies. In: Proceedings of 13th European semantic web conference, Heraklion, Crete, Greece, May 29–June 2. Springer, Berlin, pp 642–657

  27. Kirch W (ed) (2008) Pearson’s correlation coefficient. Springer, Netherlands

  28. Kotis K, Vouros GA, Alonso JP (2004) HCOME: A tool-supported methodology for engineering living ontologies. In: Proceedings of the 2nd international workshop on semantic web and databases, Toronto, Canada, August 29–30. Springer, Berlin, pp 155–166

  29. Lantow B (2016) OntoMetrics: application of on-line ontology metric calculation. In: Joint proceedings of the BIR 2016 workshops and doctoral consortium co-located with 15th international conference on perspectives in business informatics research, Prague, Czech Republic, September 14–16

  30. Ma Y, Jin B, Feng Y (2010) Semantic oriented ontology cohesion metrics for ontology-based systems. J Syst Softw 83(1):143–152

    Article  Google Scholar 

  31. Mall R (2014) Fundamentals of software engineering. PHI Learning Pvt Ltd, Delhi

    Google Scholar 

  32. McCabe TJ (1976) A complexity measure. IEEE Trans Softw Eng 4:308–320

    Article  MathSciNet  Google Scholar 

  33. Moser R, Pedrycz W, Succi G (2008) A comparative analysis of the efficiency of change metrics and static code attributes for defect prediction. In: Proceedings of the 30th international conference on software engineering, Leipzig, Germany, May 10–18. ACM, pp 181–190

  34. Muller JZ (2018) The Tyranny of Metrics. Princeton University Press, Princeton

    Book  Google Scholar 

  35. Noy N, Rector A, Hayes P, Welty C (2006) Defining n-ary relations on the semantic web. W3C working group note 12(4). Available at https://www.w3.org/TR/swbp-n-aryRelations/

  36. Orme AM, Tao H, Etzkorn LH (2006) Coupling metrics for ontology-based system. IEEE Softw 23(2):102–108

    Article  Google Scholar 

  37. Peroni S (2016) A simplified agile methodology for ontology development. In: Proceedings of the 13th OWL: experiences and directions workshop and 5th OWL reasoner evaluation workshop, Bologna, Italy, November 20. Springer, Berlin, pp 55–69

  38. Pinto HS, Staab S, Tempich C (2004) DILIGENT: Towards a fine-grained methodology for Distributed, Loosely-controlled and evolvInG. In: Proceedings of the 16th European conference on artificial intelligence, Valencia, Spain, August 22–27, vol 110, p 393

  39. Poveda-Villalón M, Gómez-Pérez A, Suárez-Figueroa MC (2014) OOPS! (OntOlogy Pitfall Scanner!): an on-line tool for ontology evaluation. Int J Seman Web Inform Syst 10(2):7–34

    Article  Google Scholar 

  40. Pressman RS (2005) Software engineering: a practitioner’s approach. Palgrave Macmillan, London

    MATH  Google Scholar 

  41. Presutti V, Daga E, Gangemi A, Blomqvist E (2009) eXtreme design with content ontology design patterns. In: Proceedings of the workshop on ontology patterns, collocated with the 8th international semantic web conference, Washington DC, USA, 25 October, CEUR Workshop series, pp 83–97

  42. Rahman F, Devanbu P (2013) How, and why, process metrics are better. In: Proceedings of the 35th international conference on software engineering, San Francisco, USA, May 18–26. IEEE, pp 432–441

  43. Ren Y, Parvizi A, Mellish C, Pan JZ, van Deemter K, Stevens R (2014) Towards competency question-driven ontology authoring. In: Proceedings of the 11th European semantic web conference, Crete, Greece, May 25–29. Springer, Berlin, pp 752–767

  44. Schober D, Tudose I, Svatek V, Boeker M (2012) OntoCheck: verifying ontology naming conventions and metadata completeness in protégé 4. J Biomed Semant 3(S-2):S4

  45. Shatnawi R, Li W (2008) The effectiveness of software metrics in identifying error-prone classes in post-release software evolution process. J Syst Softw 81(11):1868–1882

    Article  Google Scholar 

  46. Smith B, Ashburner M, Rosse C, Bard J, Bug W, Ceusters W, Goldberg LJ, Eilbeck K, Ireland A, Mungall CJ et al (2007) The OBO Foundry: coordinated evolution of ontologies to support biomedical data integration. Nat Biotechnol 25(11):1251

    Article  Google Scholar 

  47. Sommerville I (2010) Software engineering, 9th edn. Addison-Wesley, London

    MATH  Google Scholar 

  48. Suárez-Figueroa M, Gómez-Pérez A, Villazón-Terrazas B (2009) How to write and use the ontology requirements specification document. In: Proceedings of the international conference on on the move to meaningful internet systems, Ilamoura, Portugal, November 1–6. Springer, Berlin, pp 966–982

  49. Suárez-Figueroa MC, Gómez-Pérez A, Fernández-López M (2012) The NeOn methodology for ontology engineering. In: Ontology engineering in a networked world. Springer, Berlin, pp 9–34

  50. Suárez-Figueroa MC, Aguado de Cea G, Gómez-Pérez A (2013) Lights and shadows in creating a glossary about ontology engineering. Terminology 19(2):202–236

    Article  Google Scholar 

  51. Tartir S, Arpinar IB, Moore M, Sheth AP, Aleman-Meza B (2005) OntoQA: Metric-based ontology quality analysis. In: Proceedings of IEEE workshop on knowledge acquisition from distributed, autonomous, semantically heterogeneous data and knowledge sources at 2005 IEEE international conference on data mining, Houston, USA, November 27, pp 45–53

  52. Uschold M, Gruninger M (1996) Ontologies: Principles, methods and applications. Knowl Eng Rev 11(2):93–136

    Article  Google Scholar 

  53. Vrandečić D, Gangemi A (2006) Unit tests for ontologies. In: Proceedings of the 2006 international conference on the move to meaningful internet systems: OTM 2006 workshops, Montpellier, France, October 29–November 3. Springer, Berlin, pp 1012–1020

  54. Vrandečić D, Krötzsch M (2014) Wikidata: a free collaborative knowledgebase. Commun ACM 57(10):78–85

    Article  Google Scholar 

  55. Vrandečić D, Sure Y (2007) How to design better ontology metrics. Springer, Berlin, pp 311–325

  56. Wilsdon J (2016) The metric tide: independent review of the role of metrics in research assessment and management. Sage, Thousand Oaks

    Google Scholar 

  57. Yao H, Orme AM, Etzkorn L (2005) Cohesion metrics for ontology design and application. J Comput Sci 1(1):107–113

    Article  Google Scholar 

  58. Zhe Y, Zhang D, Chuan Y (2006) Evaluation metrics for ontology complexity and evolution analysis. In: Proceedings of the IEEE international conference on e-business engineering, Shanghai, China, October 24–26. IEEE Computer Society, pp 162–170

Download references

Acknowledgements

This work is partially supported by the H2020 project VICINITY: Open virtual neighbourhood network to connect intelligent buildings and smart objects (H2020-688467) and by a Predoctoral grant from the I+D+i program of the Universidad Politécnica de Madrid. We are very grateful to María Navas-Loro for her formula revisions and comments.

Author information

Authors and Affiliations

  1. Ontology Engineering Group, Escuela Técnica Superior de Ingenieros Informáticos, Universidad Politécnica de Madrid, Madrid, Spain

    Alba Fernández-Izquierdo, María Poveda-Villalón, Asunción Gómez-Pérez & Raúl García-Castro

Authors
  1. Alba Fernández-Izquierdo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. María Poveda-Villalón
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Asunción Gómez-Pérez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Raúl García-Castro
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alba Fernández-Izquierdo.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fernández-Izquierdo, A., Poveda-Villalón, M., Gómez-Pérez, A. et al. Towards metrics-driven ontology engineering. Knowl Inf Syst 63, 867–903 (2021). https://doi.org/10.1007/s10115-021-01545-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-021-01545-9

Keywords

Search

Navigation