Skip to main content
SpringerLink
Log in

An approach to measuring theory quality

  • Data Mining
  • Conference paper
  • First Online:
Book cover Advances in Knowledge Acquisition (EKAW 1996)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 1076))

  • 163 Accesses

Abstract

The quality of theories produced with the help of machine learning algorithms is usually measured in terms of accuracy and coverage. This paper reopens the issue of understandability of induced theories, which, while prominent in the early days of ML, seems to have fallen from favor in the sequel. This issue is especially relevant in the broader context of using ML as an aide in design and maintenance of knowledge bases for knowledge based systems. The guiding question is: beyond accuracy, what constitutes a good theory? An attempt at surveying relevant work in the fields of linguistics and cognitive psychology is made. The sympathetic reader will find this somewhat motivates the author's personal intuitions about the quality of a theory, hinging on understandability. These intuitions, in turn, point toward some simple criteria that may help in measuring quality. By way of consolation for those who do not share the author's intuitions, the criteria proposed here are objective in the sense that the measurements they provide may be evaluated from a number of contrary perspectives. Some empirical results are given in the context of theory restructuring: redundancy elimination and introduction of new intermediate concepts.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. L. W. Barsalou. Ad hoc categories. Memory & Cognition, 11(3):211–227, 1983.

    Google Scholar 

  2. Bernd Ulrich Biere. Verständlich-Machen. Max Niemeyer Verlag, Tübingen, 1989.

    Google Scholar 

  3. M. Bock. Wort-, Satz-, Textverarbeitung. Kohlhammer, Stuttgart, 1978.

    Google Scholar 

  4. Maurice Bruynooghe. Adding redundancy to obtain more reliable and readable Prolog programs. In Michel Van Caneghem (ed.), Proceedings of the First International Logic Programming Conference, pp. 129–133, Marseille, France, 1982. ADDP-GIA.

    Google Scholar 

  5. W. Buntine. Generalized Subsumption and its Applications to Induction and Redundancy. Artificial Intelligence, 36:149–176, 1988.

    Google Scholar 

  6. Jaime G. Carbonell. Paradigms for Machine Learning. Artificial Intelligence, 40:1–9, 1989.

    Google Scholar 

  7. Werner Emde. An Inference Engine for Representing Multiple Theories. In K. Monk (ed.), Knowledge Representation and Organization in Machine Learning, pp. 148–176. Springer, New York, Berlin, Tokyo, 1989. Also: KIT-Report Nr. 64, TU Berlin, 1988.

    Google Scholar 

  8. Werner Emde. Modellbildung, Wissensrevision und Wissensrepräsentation im Maschinellen Lernen. Informatik-Fachberichte 281. Springer Verlag, Berlin, New York, 1991. PhD thesis.

    Google Scholar 

  9. Werner Emde. Inductive Learning from very few Classified Examples. In Proc. 7th European Conference on Machine Learning (ECML-94), 1994. also available as GMD Tech. Report.

    Google Scholar 

  10. Li-Min Fu and Brace Buchanan. Learning Intermediate Concepts in Constructing a Hierarchical Knowledge Base. In Proc. 9th International Joint Conference on Artificial Intelligence, pp. 659–666, San Mateo, CA, 1985. Morgan Kaufman.

    Google Scholar 

  11. M. A. Gernsbacher. Language Comprehension as Structure Building. Lawrence Erlbaum, Hillsdale NJ, 1990.

    Google Scholar 

  12. Hans Hörmann. To Mean — To Understand: Problems of Psychological Semantics. Springer-Verlag, 1981.

    Google Scholar 

  13. Jörg-Uwe Kietz and Stefan Wrobel. Controlling the Complexity of Learning in Logic through Syntactic and Task-Oriented Models. In Stephen Muggleton (ed.), Proc. 1st Int. Workshop on ILP, pp. 107–126, Viana de Castelo, Portugal, 1991. Also in S.Muggleton (ed.), Inductive Logic Programming, Academic Press, 1992.

    Google Scholar 

  14. W. Kintsch. The Representation of Meaning in Memory. Erlbaum, Potomac, MD, 1974.

    Google Scholar 

  15. W. Kintsch and J Keenan. Reading rate and retention as a function of the number of propositions in the base structure of sentences. Cognitive Psychology, 5:257–274, 1973.

    Google Scholar 

  16. Yves Kodratoff. Guest Editor's Introduction (The Comprehensibility Manifesto). AI Communications, 7(2):83–85, 1994.

    Google Scholar 

  17. Franz von Kutschera. Elementare Logik. Wien, 1967.

    Google Scholar 

  18. Franz von Kutschera. Wissenschaftstheorie, volume II. Wilhelm Fink Verlag, München, 1972.

    Google Scholar 

  19. A. M. Lesgold. Pronominalization: a Device for Unifying Sentences in Memory. Journal of Verbal Learning and Verbal Behavior, 11:316–323, 1972.

    Google Scholar 

  20. G. Mandler. Organisation and Memory. In K. W. Spence and J. T. Spence (eds.), The Psychology of Learning and Motivation, volume 2, pp. 189–196. Academic Press, New York, 1967.

    Google Scholar 

  21. Ryszard S. Michalski. A Theory and Methodology of Inductive Learning. In Machine Learning — An Artificial Intelligence Approach, volume I, pp. 83–134. Morgan Kaufman, San Mateo, CA, 1983.

    Google Scholar 

  22. Donald Michie. The superarticulacy phenomenon in the context of software manufacture. Proceedings of the Royal Society of London, A 405:185–212, 1986.

    Google Scholar 

  23. K. Morik, S. Wrobel, Jörg-Uwe Kietz, and W. Emde. Knowledge Acquisition and Machine Learning. Academic Press, London, 1993.

    Google Scholar 

  24. Stephen Muggleton. Structuring Knowledge by Asking Questions. In Ivan Bratko and Nada Lavrač(eds.), Progress in Machine Learning—Proc. Second European Working Session on Learning (EWSL), Wilmslow, UK, 1987. Sigma Press.

    Google Scholar 

  25. Stephen Muggleton and Wray Buntine. Machine Invention of First-Order Predicates by Inverting Resolution. In Proc. Fifth Intern. Conf. on Machine Learning, San Mateo, CA, 1988. Morgan Kaufman.

    Google Scholar 

  26. Stephen Muggleton and Luc deRaedt. Inductive Logic Programming: Theory and Methods. Journal of Logic Programming, 19/20:629–680, 1994.

    Google Scholar 

  27. Stephen Muggleton and Cao Feng. Efficient Induction of Logic Programs. In Proc. First Conf. on Algorithmic Learning Theory, Tokyo, 1990. Ohmsha Publishers.

    Google Scholar 

  28. Claire Nedellec (ed.). Proc. IJCAI Workshop on Machine Learning and Comprehensibility, Claire.Nedellec@lri.fr, 1995.

    Google Scholar 

  29. G. Piatetsky-Shapiro and W. Frawley. Knowledge discovery in databases. The MIT press, 1991. (editors).

    Google Scholar 

  30. Gordon D. Plotkin. A note on inductive generalization. In B. Meltzer and D. Michie (eds.), Machine Intelligence, volume 5, chapter 8, pp. 153–163. American Elsevier, 1970.

    Google Scholar 

  31. Karl Popper. Die beiden Grundprobleme der Erkenntnistheorie: aufgrund von Ms. aus d. Jahren 1930–1933, volume 18 of Die Einheit der Gesellschaftswissenschaften. Mohr, Tübingen, 1933. edited by Troels Eggers Hansen, appeared 1979.

    Google Scholar 

  32. J. Ross Quinlan. Learning Logical Definitions from Relations. Machine Learning, 5(3):239–266, 1990.

    Google Scholar 

  33. A. J. Sanford and S. C. Garrod. Understanding Written Language. Wiley and Sons, Chichester, 1981.

    Google Scholar 

  34. P. Smolensky. Connectionist AI, Symbolic AI, and the Brain. Artificial Intelligence Review, 1:95–109, 1987.

    Google Scholar 

  35. E. Sommer, K. Morik, J.M. Andre, and M. Uszynski. What On-line Learning Can Do for Knowledge Acquisition. Knowledge Acquisition, 6:435–460, 1994.

    Google Scholar 

  36. E. Sommer, Werner Emde, Jörg-Uwe Kietz, and Stefan Wrobel. Mobal 42 User Guide ((always) Draft). Arbeitspapiere der gmd, GMD, 1996. Available via WWW http://nathan.gmd.de/projects/ml/lit/mlpublist.html.

    Google Scholar 

  37. E. Sommer. Learning Relations without Closing the World. In Proc. of the European Conference on Machine Learning (ECML-94), Berlin, 1994. Springer-Verlag.

    Google Scholar 

  38. E. Sommer. Restructuring in Horn clause knowledge bases. Technical report, ESPRIT Project ILP (6020), 1994. ILP Deliverable GMD 2.1.

    Google Scholar 

  39. E. Sommer. Rulebase Stratification: an Approach to theory restructuring. In Proc. 4th Intl. Workshop on Inductive Logic Programming (ILP-94), 1994. Available via WWW http://nathan.gmd.de/projects/ml/lit/mlpublist.html.

    Google Scholar 

  40. E. Sommer. An Approach to Quantifying the Quality of Induced Theories. In Claire Nedellec (ed.), Proc. IJCAI Workshop on Machine Learning and Comprehensibility, 1995. Available via WWW http: //nathan.gmd.de/projects/ml/lit/mlpublist. html.

    Google Scholar 

  41. E. Sommer. Fender: An approach to theory restructuring. In Stefan Wrobel and Nada Lavrac (eds.), Proc. of the European Conference on Machine Learning (ECML-95), volume 912 of Lecture Notes in Artificial Intelligence, Berlin, 1995. Springer-Verlag.

    Google Scholar 

  42. E. Sommer. Mobal's theory restructuring tool RRT. Technical report, ESPRIT Project ILP (6020), 1995. ILP Deliverable GMD 2.2.

    Google Scholar 

  43. E. Sommer. Theory Restructuring. NN, 1996. (submitted).

    Google Scholar 

  44. Stefan Wrobel. Concept Formation and Knowledge Revision. Kluwer Academic Publishers, Dordrecht, Netherlands, 1994.

    Google Scholar 

  45. L. S. Wygotski. Denken und Sprechen. Conditio humana. S. Fischer, 1964. (First published in Russian 1934, english translation “Thought and language” 1962 by MIT Press).

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. AI Division (I3.KI), GMD (German National Research Center for Information Technology), Schloss Birlinghoven, 53757, Sankt Augustin, Germany

    Edgar Sommer

Authors
  1. Edgar Sommer
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Nigel Shadbolt Kieron O'Hara Guus Schreiber

Rights and permissions

Reprints and permissions

Copyright information

© 1996 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Sommer, E. (1996). An approach to measuring theory quality. In: Shadbolt, N., O'Hara, K., Schreiber, G. (eds) Advances in Knowledge Acquisition. EKAW 1996. Lecture Notes in Computer Science, vol 1076. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-61273-4_13

Download citation

  • DOI: https://doi.org/10.1007/3-540-61273-4_13

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-61273-5

  • Online ISBN: 978-3-540-68391-9

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation