Skip to main content
SpringerLink
Log in

The Evaluation of Ontologies for Quality, Suitability for Reuse, and the Significant Role of Social Factors

  • Conference paper
  • First Online:
Book cover Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K 2018)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1222))

Abstract

A frequent challenge faced by ontologists and knowledge engineers is the choice of the correct or most appropriate ontology for reuse. Despite the importance of ontology evaluation and selection and the widespread research on these topics, there are still many unanswered questions and challenges. Most of the evaluation metrics and frameworks in the literature are mainly based on a limited set of internal characteristics of ontologies, e.g., their content and structure, which ignore how the community uses and evaluates them. This paper used a survey questionnaire to investigate the notion of quality and reusability in ontology engineering, and to explore and identify the set of metrics that can affect the process of ontology evaluation and selection for reuse. Responses from 157 ontologists and knowledge engineers were collected, and their analysis suggests that the process of ontology evaluation and selection for reuse, not only depends on different internal characteristics of ontologies, but that it also depends on different metadata, and social and community related metrics. Findings of this research can contribute to facilitating and improving the process of selecting an ontology for reuse.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bontas, E.P., Mochol, M., Tolksdorf, R.: Case studies on ontology reuse. In: Proceedings of the IKNOW05 International Conference on Knowledge Management, vol. 74, p. 345, June 2005

    Google Scholar 

  2. d’Aquin, M., et al.: What can be done with the semantic web? An overview of watson-based applications. In: CEUR Workshop Proceedings, vol. 426 (2008)

    Google Scholar 

  3. Fernández, M., Cantador, I., Castells, P.: CORE: a tool for collaborative ontology reuse and evaluation. In: CEUR Workshop Proceedings, vol. 179 (2006)

    Google Scholar 

  4. Gómez-Pérez, A.: Some ideas and examples to evaluate ontologies. In: 11th Conference on Artificial Intelligence for Applications Proceedings, pp. 299–305. IEEE, February 1995

    Google Scholar 

  5. Sabou, M., et al.: Ontology selection: ontology evaluation on the real semantic web. In: Vrandecic, D., et al. (ed.) 4th International EON Workshop, Evaluation of Ontologies for the Web, EON 2006, (CEUR Workshop Proceedings), Edinburgh, Scotland, UK (2006)

    Google Scholar 

  6. Talebpour, M., Sykora, M.D., Jackson, T.: Ontology selection for reuse: will it ever get easier?. In: 10th International Conference on Knowledge Engineering and Ontology Development, 18–20 September, Seville, Spain (2018)

    Google Scholar 

  7. Gómez-Pérez, A., Facultad, D.I.: Evaluation of taxonomic knowledge in ontologies and knowledge bases (1999). http://sern.ucalgary.ca/KSI/KAW/KAW99

  8. Hlomani, H., Stacey, D.: Approaches, methods, metrics, measures, and subjectivity in ontology evaluation: a survey. Semant. Web J. 1, 1–11 (2014). http://www.semantic-web-journal.net/system/files/swj657.pdf

    Google Scholar 

  9. Suárez-Figueroa, M.C., Gómez-Pérez, A.: First attempt towards a standard glossary of ontology engineering terminology. In: 8th International Conference on Terminology and Knowledge Engineering, TKE 2008 (2008). http://www.neon-project.org/. Accessed 17 Feb 2019

  10. Gómez-Pérez, A.: From knowledge based systems to knowledge sharing technology: evaluation and assessment, differences, pp. 1–15 (1994). http://oa.upm.es/6498/

  11. Brank, J., Mladenic, D., Grobelnik, M.: Gold standard based ontology evaluation using instance assignment. In: Workshop on Evaluation of Ontologies for the Web, EON, May 2006

    Google Scholar 

  12. Yu, J., Thom, J.A., Tam, A.: Requirements-oriented methodology for evaluating ontologies. Inf. Syst. 34(8), 766–791 (2009)

    Article  Google Scholar 

  13. Tartir, S., Arpinar, I.B., Sheth, A.P.: Ontological evaluation and validation. In: Poli, R., Healy, M., Kameas, A. (eds.) Theory and Applications of Ontology: Computer Applications. Springer, Dordrecht (2010). https://doi.org/10.1007/978-90-481-8847-5_5

    Chapter  Google Scholar 

  14. Ning, H., Shihan, D.: Structure-based ontology evaluation. In: 2006 IEEE International Conference on e-Business Engineering, ICEBE 2006, pp. 132–137. IEEE, October 2006

    Google Scholar 

  15. Brank, J., Grobelnik, M., Mladenic, D.: A survey of ontology evaluation techniques. In: Conference on Data Mining and Data Warehouses, SiKDD 2005, Ljubljana, Slovenia, p. 4 (2005)

    Google Scholar 

  16. Arpinar, I.B., Giriloganathan, K., Aleman-Meza, B.: Ontology quality by detection of conflicts in metadata. In: Proceedings of the 4th International EON Workshop, May 2006

    Google Scholar 

  17. Bandeira, J., et al.: FOCA: a methodology for ontology evaluation, vol. 3, pp. 1–3 (2016). http://arxiv.org/abs/1612.03353

  18. Maiga, G., Ddembe, W.: A flexible approach for user evaluation of biomedical ontologies (2008)

    Google Scholar 

  19. Brewster, C., et al.: Data driven ontology evaluation. In: 4th International Conference on Language Resources and Evaluation, LREC 2004, Lisbon, Portugal, p. 4 (2004)

    Google Scholar 

  20. Gangemi, A., Catenacci, C., Ciaramita, M., Lehmann, J.: Modelling Ontology Evaluation and Validation. In: Sure, Y., Domingue, J. (eds.) ESWC 2006. LNCS, vol. 4011, pp. 140–154. Springer, Heidelberg (2006). https://doi.org/10.1007/11762256_13

    Chapter  Google Scholar 

  21. Obrst, L., Ceusters, W., Mani, I., Ray, S., Smith, B.: The evaluation of ontologies. In: Baker, C.J.O., Cheung, K.H. (eds.) Semantic Web, pp. 139–158. Springer, Boston (2007). https://doi.org/10.1007/978-0-387-48438-9_8

    Chapter  Google Scholar 

  22. Yu, J., Thom, J.A., Tam, A.: Ontology evaluation using wikipedia categories for browsing. In: Proceedings of the Sixteenth ACM Conference on Information and Knowledge Management, pp. 223–232. ACM, November 2007

    Google Scholar 

  23. McDaniel, M., Storey, V.C., Sugumaran, V.: The role of community acceptance in assessing ontology quality. In: Métais, E., Meziane, F., Saraee, M., Sugumaran, V., Vadera, S. (eds.) NLDB 2016. LNCS, vol. 9612, pp. 24–36. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-41754-7_3

    Chapter  Google Scholar 

  24. Supekar, K.: A peer-review approach for ontology evaluation. In: 8th International Protege Conference, pp. 77–79, July 2005

    Google Scholar 

  25. Maedche, A., Staab, S.: Measuring similarity between ontologies. In: Gómez-Pérez, A., Benjamins, V.R. (eds.) EKAW 2002. LNCS (LNAI), vol. 2473, pp. 251–263. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-45810-7_24

    Chapter  MATH  Google Scholar 

  26. Fahad, M., Qadir, M.A.: A framework for ontology evaluation. ICCS Suppl. 354, 149–158 (2008)

    Google Scholar 

  27. Netzer, Y., Gabay, D., Adler, M., Goldberg, Y., Elhadad, M.: Ontology evaluation through text classification. In: Chen, L., et al. (eds.) APWeb/WAIM -2009. LNCS, vol. 5731, pp. 210–221. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-03996-6_20

    Chapter  Google Scholar 

  28. Martínez-Romero, M., Jonquet, C., O’connor, M.J., Graybeal, J., Pazos, A., Musen, M.A.: NCBO ontology recommender 2.0: an enhanced approach for biomedical ontology recommendation, J. Biomed. Semant., 8(1), 21 (2017)

    Google Scholar 

  29. Porzel, R., Malaka, R.: A task-based approach for ontology evaluation. In: ECAI Workshop on Ontology Learning and Population, Valencia, Spain, pp. 1–6, August 2004

    Google Scholar 

  30. Kehagias, D.D., Papadimitriou, I., Hois, J., Tzovaras, D., Bateman, J.: A methodological approach for ontology evaluation and refinement. In: ASK-IT Final Conference. June (Cit. on p.), pp. 1–13, June 2008

    Google Scholar 

  31. Burton-Jones, A., Storey, V.C., Sugumaran, V., Ahluwalia, P.: A semiotic metrics suite for assessing the quality of ontologies. Data Knowl. Eng. 55(1), 84–102 (2005)

    Article  Google Scholar 

  32. Buitelaar, P., Eigner, T., Declerck, T.: OntoSelect: a dynamic ontology library with support for ontology selection. In: Proceedings of the Demo Session at the International Semantic Web Conference (2004)

    Google Scholar 

  33. Buitelaar, P., Eigner, T.: Ontology search with the ontoselect ontology library. In: LREC, May 2008

    Google Scholar 

  34. Talebpour, M., Sykora, M. Jackson, T.W.: The role of community and social metrics in ontology evaluation: an interview study of ontology reuse. In: 9th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management, pp. 119–127 (2017) https://doi.org/10.5220/0006589201190127

  35. Fernández, M., Overbeeke, C., Sabou, M., Motta, E.: What makes a good ontology? A case-study in fine-grained knowledge reuse. In: Gómez-Pérez, A., Yu, Y., Ding, Y. (eds.) ASWC 2009. LNCS, vol. 5926, pp. 61–75. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-10871-6_5

    Chapter  Google Scholar 

  36. Gill, T.: Metadata and the Web: Introduction to metadata, vol. 3, pp. 20–38. Getty publications, Los Angeles (2008)

    Google Scholar 

  37. Sowa, J.F.: Ontology, metadata, and semiotics. In: Ganter, B., Mineau, G.W. (eds.) ICCS-ConceptStruct 2000. LNCS (LNAI), vol. 1867, pp. 55–81. Springer, Heidelberg (2000). https://doi.org/10.1007/10722280_5

    Chapter  Google Scholar 

  38. Lozano-Tello, A., Gómez-Pérez, A.: Ontometric: a method to choose the appropriate ontology. J. Database Manage. (JDM) 15(2), 1–18 (2004)

    Article  Google Scholar 

  39. Ding, L.: Swoogle: a search and metadata engine for the semantic web. In: Proceedings of the Thirteenth ACM International Conference on Information and Knowledge Management, pp. 652–659. ACM, November 2004

    Google Scholar 

  40. Hartmann, J., Sure, Y., Haase, P., Palma, R., Suarez-Figueroa, M.: OMV–ontology metadata vocabulary. In: ISWC, vol. 3729, November 2005

    Google Scholar 

  41. Matentzoglu, N., Malone, J., Mungall, C., Stevens, R.: MIRO: guidelines for minimum information for the reporting of an ontology. J. Biomed. Semant. 9(1), 6 (2018)

    Article  Google Scholar 

  42. Lewen, H., d’Aquin, M.: Extending open rating systems for ontology ranking and reuse. In: Cimiano, P., Pinto, H.S. (eds.) EKAW 2010. LNCS (LNAI), vol. 6317, pp. 441–450. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-16438-5_34

    Chapter  Google Scholar 

  43. Lewen, H., Supekar, K., Noy, N.F., Musen, M.A.: Topic-specific trust and open rating systems: an approach for ontology evaluation. In; Workshop on Evaluation of Ontologies for the Web, May 2006

    Google Scholar 

  44. Morse, J.M.: Mixed Method Design: Principles and Procedures. Routledge, London (2016)

    Book  Google Scholar 

  45. Talebpour, M., Sykora, M., Jackson, T.: Social and community related themes in ontology evaluation: findings from an interview study. In: Fred, A., et al. (eds.) IC3K 2017. CCIS, vol. 976, pp. 320–336. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-15640-4_16

    Chapter  Google Scholar 

  46. Supekar, K., Patel, C., Lee, Y., Characterizing Quality of Knowledge on Semantic Web. In: FLAIRS Conference, pp. 472–478, May 2004

    Google Scholar 

  47. Wang, X., Guo, L., Fang, J.: Automated ontology selection based on description logic. In: Proceedings 2008 12th International Conference on Computer Supported Cooperative Work in Design, CSCWD, vol. 1, pp. 482–487 (2008)

    Google Scholar 

  48. Martínez-Romero, M., Vázquez-Naya, J.M., Pereira, J., Pazos, A.: BiOSS: a system for biomedical ontology selection. Comput. Methods Programs Biomed. 114(1), 125–140 (2014)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Business and Economics, Loughborough University, Loughborough, UK

    Marzieh Talebpour, Martin Sykora & Tom Jackson

Authors
  1. Marzieh Talebpour
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Martin Sykora
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tom Jackson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marzieh Talebpour .

Editor information

Editors and Affiliations

  1. Instituto de Telecomunicações, University of Lisbon, Lisbon, Portugal

    Ana Fred

  2. Federal University of Pernambuco, Recife, Brazil

    Ana Salgado

  3. University of Madeira, Funchal, Portugal

    David Aveiro

  4. Delft University of Technology, Delft, The Netherlands

    Jan Dietz

  5. University of Coimbra, Coimbra, Portugal

    Jorge Bernardino

  6. Polytechnic Institute of Setúbal/INSTIC, Setúbal, Portugal

    Joaquim Filipe

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Talebpour, M., Sykora, M., Jackson, T. (2020). The Evaluation of Ontologies for Quality, Suitability for Reuse, and the Significant Role of Social Factors. In: Fred, A., Salgado, A., Aveiro, D., Dietz, J., Bernardino, J., Filipe, J. (eds) Knowledge Discovery, Knowledge Engineering and Knowledge Management. IC3K 2018. Communications in Computer and Information Science, vol 1222. Springer, Cham. https://doi.org/10.1007/978-3-030-49559-6_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-49559-6_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-49558-9

  • Online ISBN: 978-3-030-49559-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation