Evaluation of Ontologies and DL Reasoners

  • Muhammad Fahad
  • Muhammad Abdul Qadir
  • Syed Adnan Hussain Shah
Part of the IFIP – The International Federation for Information Processing book series (IFIPAICT, volume 288)


Ontology driven architecture has revolutionized the inference system by allowing interoperability and efficient reasoning between heterogeneous multi-vendors systems. Sound reasoning support is highly important for sound semantic web ontologies which can only be possible if state-of-the-art Description Logic Reasoners were capable enough to identify inconsistency and classify taxonomy in ontologies. We have discussed existing ontological errors and design anomalies, and provided a case study incorporating these errors. We have evaluated consistency, subsumption, and satisfiability of DL reasoners on the case study. Experiment with DL reasoners opens up number of issues that were not incorporated within their followed algorithms. Especially circulatory errors and various types of semantic inconsistency errors that may cause serious side effects need to be detected by DL reasoners for sound reasoning from ontologies. The evaluation of DL reasoners on Automobile ontology helps in updating the subsumption, satisfiability and consistency checking algorithms for OWL ontologies, especially the new constructs of OWL 1.1.


Circulatory Error Property Hierarchy Datatype Property Tableau Algorithm Ontology Evaluation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    G. Antoniou, and F.V. Harmelen, A Semantic Web Primer. MIT Press Cambridge, ISBN 0-262-01210-3, 2004.Google Scholar
  2. 2.
    J. Baumeister, and D.S. Seipel, Owls—Design Anomalies in Ontologies, 18th Intl. Floida Artificial Intelligence Research Society Conference (FLAIRS), pp 251–220, 2005.Google Scholar
  3. 3.
    C. Brewster et al, Data driven ontology evaluation. Proceedings of Intl. Conf. on Language Resources and Evaluation, Lisbon, 2004.Google Scholar
  4. 4.
    M. Fahad, M.A. Qadir, M.W. Noshairwan, N. Iftikhar,. DKP-OM: A Semantic Based Ontology Merger. In Proc. 3rd International conference on Semantic Technologies, I-Semantics 5–7 September 2007, Journal of Universal Computer Science (J.UCS). 2007aGoogle Scholar
  5. 5.
    M. Fahad, M.A. Qadir, W. Noshairwan, Semantic Inconsistency Errors in Ontologies. Proc. of GRC 07, Silicon Valley USA. IEEE CS. pp 283–286, 2007b.Google Scholar
  6. 6.
    A. Gomez-Perez, Some ideas and examples to evaluate ontologies. KSL, Stanford University., 1994.Google Scholar
  7. 7.
    A. Gomez-Perez, M.F. Lopez, and O.C. Garcia, Ontological Engineering: With Examples from the Areas of Knowledge Management, E-Commerce and the Semantic Web. Springer ISBN:1-85253-55j-3, 2001.Google Scholar
  8. 8.
    A. Gomez-Perez et al., Evaluation of Taxonomic Knowledge on Ontologies and Knowledge-Based Systems. Intl. Workshop on Knowledge Acquisition, Modeling and Management., 1999.Google Scholar
  9. 9.
    C. Jelmini, and S. M-Maillet, OWL-based reasoning with retractable inference”, In RIAO Conference Proceedings 2004.Google Scholar
  10. 10.
    A. Maedche and S. Staab, Measuring similarity betwe- en ontologies. Proc. CIKM 2002. LNAI vol. 2473, 2002.Google Scholar
  11. 11.
    D. Nardi, et al. 2000. The Description Logic Handbook: Theory, Implementation, and Applications.Google Scholar
  12. 12.
    W. Noshairwan, M.A. Qadir, M.A., M. Fahad, Sufficient Knowledge Omission error and Redundant Disjoint Relation in Ontology. InProc. 5th Atlantic Web Intelligence Conference June 25–27, France, 2007a.Google Scholar
  13. 13.
    R. Porzel, R. Malaka, A task-based approach for ontology evaluation. ECAI Workshop Ont. Learning and Population, 2004.Google Scholar
  14. 14.
    M.A. Qadir, W. Noshairwan, Warnings for Disjoint Knowledge Omission in Ontologies. Second International Conference on internet and Web Applications and Services (ICIW07). IEEE, p. 45, 2007a.Google Scholar
  15. 15.
    M.A. Qadir, M. Fahad, S.A.H. Shah, Incompleteness Errors in Ontologies. Proc. of Intl GRC 07, USA. IEEE Computer Society. pp 279–282, 2007b.Google Scholar
  16. 16.
    K. Supekar, A peer-review approach for ontology evaluation. Proc. 8th Intl. Protégé Conference, Madrid, Spain, July 18–21, 2005.Google Scholar
  17. 17.
    M. Fahad, and M.A. Qadir, A Framework for ontology evaluation. 16th Intl. Proceeding of Conceptual Structures. July 2008, France. Vol-354, pages 149–158, 2008a.Google Scholar
  18. 18.
    M. Fahad, M.A. Qadir, M.W. Noshairwan, Ontological Errors: Inconsistency, Incompleteness and Redundancy. (to appear) In proc. 10th International Conference on Enterprise Information Systems (ICEIS’08). June 2008. Barcelona, Spain, 2008b.Google Scholar
  19. 19.
    V. Haarslev, R. Möller, Racer system description. In Gor’e, R., Leitsch, A., Nipkow, T., eds.: International Joint Conference on Automated Reasoning, IJCAR’ 2001, June 18–23, Siena, Italy, Springer-Verlag (2001) 701–705Google Scholar
  20. 20.
    Z. Pan, Benchmarking DL Reasoners Using Realistic Ontologies. Bell Labs Research and Lehigh University, 2007Google Scholar
  21. 21.
    B. Parsia, E. Sirin, Pellet: An owl dl reasoner. In: Proc. International Semantic Web Conference. (2005)Google Scholar
  22. 22.
    I. Horrocks, U. Sattler, A tableaux decision procedure for SHOIQ. In: Proceedings of Nineteenth International Joint Conference on Artificial Intelligence. (2005)Google Scholar
  23. 23.
    I. Horrocks, The FaCT System. International conference. on Analytic Tableaux and Related Methods (TABLEAUX’98), pp 307–312,vol 1397, Springer-Verlag, 1998Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2008

Authors and Affiliations

  • Muhammad Fahad
    • 1
  • Muhammad Abdul Qadir
    • 1
  • Syed Adnan Hussain Shah
    • 1
  1. 1.Mohammad Ali Jinnah UniversityIslamabadPakistan

Personalised recommendations