Evaluation of Ontology Enhancement Tools

  • Myra Spiliopoulou
  • Markus Schaal
  • Roland M. Müller
  • Marko Brunzel
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4289)


Mining algorithms can enhance the task of ontology establishment but methods are needed to assess the quality of their findings. Ontology establishment is a long-term interactive process, so it is important to evaluate the contribution of a mining tool at an early phase of this process so that only appropriate tools are used in later phases. We propose a method for the evaluation of such tools on their impact on ontology enhancement. We model impact as quality perceived by the expert and as statistical quality computed by an objective function. We further provide a mechanism that juxtaposes the two forms of quality. We have applied our method on an ontology enhancement tool and gained some interesting insights on the interplay between perceived impact and statistical quality.


Statistical Quality Application Domain Domain Expert Semantic Annotation Cluster Label 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Cimiano, P., Staab, S., Tane, J.: Automatic acquisition of taxonomies from text: Fca meets nlp. In: Proc. of the ECML/PKDD Workshop on Adaptive Text Extraction and Mining, Cavtat, Croatia, September 2003, pp. 10–17 (2003)Google Scholar
  2. 2.
    Dill, S., Eiron, N., Gibson, D., Gruhl, D., Guha, R., Jhingran, A., Kanungo, T., Rajagopalan, S., Tomkins, A., Tomlin, J.A., Zien, J.Y.: SemTag and seeker: Bootstrapping the semantic web via automated semantic annotation. In: Proc. of the 12th Int. World Wide Web Conf., Budapest, Hungary, pp. 178–186. ACM Press, New York (2003)Google Scholar
  3. 3.
    Faatz, A., Steinmetz, R.: Precision and recall for ontology enrichment. In: Proc. of ECAI-2004 Workshop on Ontology Learning and Population, Valencia, Spain (August 2004), (accessed at July 26, 2005)
  4. 4.
    Faure, D., Nédellec, C.: Knowledge acquisition of predicate argument structures from technical texts using machine learning: The system ASIUM. In: Fensel, D., Studer, R. (eds.) EKAW 1999. LNCS (LNAI), vol. 1621, pp. 329–334. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  5. 5.
    Haase, P., Hotho, A., Schmidt-Thieme, L., Sure, Y.: Collaborative and usage-driven evolution of personal ontologies. In: Gómez-Pérez, A., Euzenat, J. (eds.) ESWC 2005. LNCS, vol. 3532, pp. 486–499. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  6. 6.
    Handschuh, S., Staab, S., Ciravegna, F.: S-CREAM – Semi-automatic CREation of metadata. In: Proc. of the European Conf. on Knowledge Acquisition and Management (2002)Google Scholar
  7. 7.
    Holsapple, C., Joshi, K.D.: A collaborative approach to ontology design. Communications of ACM 45(2), 42–47 (2005)Google Scholar
  8. 8.
    Hotho, A., Staab, S., Stumme, G.: Explaining text clustering results using semantic structures. In: Lavrač, N., Gamberger, D., Todorovski, L., Blockeel, H. (eds.) PKDD 2003. LNCS (LNAI), vol. 2838, pp. 217–228. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  9. 9.
    Kavalec, M., Svatek, V.: A study on automated relation labelling in ontology learning. In: Buitelaar, P., Cimiano, P., Magnini, B. (eds.) Ontology Learning and Population. IOS Press, Amsterdam (2005)Google Scholar
  10. 10.
    Jianming, L., Zhang, L., Yong, Y.: Learning to generate semantic annotation for domain specific sentences. In: Proc. of the ”Knowledge Markup and Semantic Annotation” Workshop of the K-CAP 2001 Conference (2001)Google Scholar
  11. 11.
    Maedche, A., Staab, S.: Semi-automatic engineering of ontologies from text. In: Proc. of 12th Int. Conf. on Software and Knowledge Engineering, Chicago, IL (2000)Google Scholar
  12. 12.
    Navigli, R., Velardi, P., Cucchiarelli, A., Neri, F.: Quantitative and qualitative evaluation of the ontolearn ontology learning system. In: Proc. of ECAI-2004 Workshop on Ontology Learning and Population, Valencia, Spain (August 2004), (accessed at July 26, 2005)
  13. 13.
    Porzel, R., Malaka, R.: A task-based approach for ontology evaluation. In: Proc. of ECAI-2004 Workshop on Ontology Learning and Population, Valencia, Spain, (August 2004), (accessed at July 26, 2005)
  14. 14.
    Schaal, M., Mueller, R., Brunzel, M., Spiliopoulou, M.: RELFIN - topic discovery for ontology enhancement and annotation. In: Gómez-Pérez, A., Euzenat, J. (eds.) ESWC 2005. LNCS, vol. 3532, pp. 608–622. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  15. 15.
    Stein, B., zu Eissen, S.M., Wißbrock, F.: On Cluster Validity and the Information Need of Users. In: Hanza, M.H. (ed.) 3rd IASTED Int. Conference on Artificial Intelligence and Applications (AIA 2003), Benalmadena, Spain, September 2003, pp. 216–221. ACTA Press (2003)Google Scholar
  16. 16.
    Vazirgiannis, M., Halkidi, M., Gunopoulos, D.: Uncertainty Handling and Quality Assessment in Data Mining. Springer, Heidelberg (2003)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Myra Spiliopoulou
    • 1
  • Markus Schaal
    • 2
  • Roland M. Müller
    • 1
  • Marko Brunzel
    • 1
  1. 1.Otto-von-Guericke-Universität Magdeburg 
  2. 2.Bilkent UniversityAnkara

Personalised recommendations