Learning = Inferencing + Memorizing

Basic Concepts of Inferential Theory of Learning and Their Use for Classifying Learning Processes
  • Ryszard S. Michalski
Part of the The Springer International Series in Engineering and Computer Science book series (SECS, volume 195)


This chapter presents a general conceptual framework for describing and classifying learning processes. The framework is based on the Inferential Theory of Learning that views learning as a search through a knowledge space aimed at deriving knowledge that satisfies a learning goal. Such a process involves performing various forms of inference, and memorizing results for future use. The inference may be of any type—deductive, inductive or analogical. It can be performed explicitly, as in many symbolic systems, or implicitly, as in artificial neural nets. Two fundamental types of learning are distinguished: analytical learning that reformulates a given knowledge to the desirable form (e.g., skill acquisition), and synthetic learning that creates new knowledge (e.g., concept learning). Both types can be characterized in terms of knowledge transmutations that are involved in transforming given knowledge (input plus background knowledge) into the desirable knowledge. Several transmutations are discussed in a novel way, such as deductive and inductive generalization, abductive derivation, deductive and inductive specialization, abstraction and concretion. The presented concepts are used to develop a general classification of learning processes.

Key words

learning theory machine learning inferential theory of learning deduction induction abduction generalization abstraction knowledge transmutation classification of learning 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Adler, M. J., Gorman, W. (Eds.,) The Great Ideas: A Synopicon of Great Books of the Western World, Vol. 1, Ch. 39 (Induction), pp. 565–571, Encyclopedia Britannica, Inc., 1987.Google Scholar
  2. Aristotle, Posterior Analytics, in The Works of Aristotle, Volume 1, R. M. Hutchins (Ed.), Encyclopedia Britannica, Inc., 1987.Google Scholar
  3. Bacon, F., Novum Organum, 1620.Google Scholar
  4. Bareiss, E. R., Porter, B. and Wier, C.C., PROTOS, An Exemplar-based Learning Apprentice, in Machine Learning: An Artificial Intelligence Approach, Vol. III, Carbonell, J.G., and Mitchell, T. M. (Eds.), Morgan Kaufmann, 1990.Google Scholar
  5. Bergadano, F., Matwin, S., Michalski, R.S. and Zhang, J., Learning Two-tiered Descriptions of Flexible Concepts: The POSEIDON System, Machine Learning Journal, Vol. 8,No, 1, Januray 1992.Google Scholar
  6. Carbonell, J. G., Michalski R.S. and Mitchell, T.M., An Overview of Machine Learning, in Machine Learning: An Artificial Intelligence Approach, Michalski, R.S., Carbonell, J.G., and Mitchell, T. M. (Eds.), Morgan Kaufmann Publishers, 1983.Google Scholar
  7. Cohen, L.J., The Implications of Induction, London, 1970.Google Scholar
  8. Collins, A. and Michalski, R.S., “The Logic of Plausible Reasoning: A Core Theory,” Cognitive Science, Vol. 13, pp. 1–49, 1989.CrossRefGoogle Scholar
  9. Danyluk. A. P., “Recent Results in the Use of Context for Learning New Rules,” Technical Report No. TR-98-066, Philips Laboratories, 1989.Google Scholar
  10. DeJong, G. and Mooney, R., “Explanation-Based Learning: An Alternative View,” Machine Learning Journal, Vol 1,No. 2, 1986.Google Scholar
  11. Dietterich, T.G., and Flann, N.S., “An Inductive Approach to Solving the Imperfect Theory Problem,” Proceedings of 1988 Symposium on Explanation-Based Learning, pp. 42–46, Stanford University, 1988.Google Scholar
  12. Goodman, L.A. and Kruskal, W.H., Measures of Association for Cross Classifications, Springer-Verlag, New York, 1979.zbMATHGoogle Scholar
  13. Kodratoff, Y., and Tecuci, G., “DISCIPLE-1: Interactive Apprentice System in Weak Theory Fields,” Proceedings of IJCAI-87, pp. 271–273, Milan, Italy, 1987.Google Scholar
  14. Lebowitz, M., “Integrated Learning: Controlling Explanation,” Cognitive Science, Vol. 10,No. 2, pp. 219–240, 1986.CrossRefGoogle Scholar
  15. Michalski, R.S., “A Planar Geometrical Model for Representing Multi-Dimensional Discrete Spaces and Multiple-Valued Logic Functions,” Report No. 897, Department of Computer Science, University of Illinois, Urbana, January 1978.Google Scholar
  16. Michalski, R. S., “Theory and Methodology of Inductive Learning,” Machine Learning: An Artificial Intelligence Approach, R. S. Michalski, J. G. Carbonell, T. M. Mitchell (Eds.), Tioga Publishing Co., 1983.Google Scholar
  17. Michalski, R.S., Understanding the Nature of Learning: Issues and Research Directions, in Machine Learning: An Artificial Intelligence Approach Vol. II, Michalski, R.S., Carbonell, J.G., and Mitchell, T. M. (Eds.), Morgan Kaufmann Publishers, 1986.Google Scholar
  18. Michalski, R.S., Toward a Unified Theory of Learning: Multistrategy Task-adaptive Learning, Reports of Machine Learning and Inference Laboratory MLI-90-I, January 1990a.Google Scholar
  19. Michalski, R.S. and Kodratoff, Y. “Research in Machine Learning: Recent Progress, Classification of Methods and Future Directions,” in Machine Learning: An Artificial Intelligence Approach, Vol. III, Kodratoff, Y. and Michalski, R.S. (eds.), Morgan Kaufmann Publishers, Inc., 1990b.Google Scholar
  20. Michalski, R.S., LEARNING FLEXIBLE CONCEPTS: Fundamental Ideas and a Method Based on Two-tiered Representation, in Machine Learning: An Artificial Intelligence Approach, Vol. III, Kodratoff, Y. and Michalski, R.S. (eds.), Morgan Kaufmann Publishers, Inc., 1990c.Google Scholar
  21. Michalski, R.S., INFERENTIAL THEORY OF LEARNING: Developing Foundations for Multistrategy Learning, in Machine Learning: A Multistrategy Approach, Vol. IV, R.S. Michalski and G. Tecuci (Eds.), Morgan Kaufmann, 1993.Google Scholar
  22. Minton, S., “Quantitative Results Concerning the Utility of Explanation-Based Learning,” Proceedings of AAAI-88, pp. 564–569, Saint Paul, MN, 1988.Google Scholar
  23. Minton, S., Carbonell, J.G., Etzioni, O., et al., “Acquiring Effective Search Control Rules: Explanation-Based Learning in the PRODIGY System,” Proceedings of the 4th International Machine Learning Workshop, pp. 122–133, University of California, Irvine, 1987.Google Scholar
  24. Mitchell, T.M., Keller, T., Kedar-Cabelli, S., “Explanation-Based Generalization: A Unifying View,” Machine Learning Journal, Vol. 1, January 1986.Google Scholar
  25. Pazzani, M.J., “Integrating Explanation-Based and Empirical Learning Methods in OCCAM,” Proceedings of EWSL-88, pp. 147–166, Glasgow, Scotland, 1988.Google Scholar
  26. Pearl J., Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, Morgan Kaufmann, 1988.Google Scholar
  27. Piatetsky-Shapiro, G., “Probabilistic Data Dependencies,” Proceedings of the ML92 Workshop on Machine Discovery, J.M. Zytkow (Ed.), Aberdeen, Scotland, July 4, 1992.Google Scholar
  28. Popper, K. R., Objective Knowledge: An Evolutionary Approach, Oxford at the Clarendon Press, 1972.Google Scholar
  29. Poole, D., Explanation and Prediction: An Architecture for Default and Abductive Reasoning, Computational Intelligence, No. 5, pp. 97–110, 1989.CrossRefGoogle Scholar
  30. Porter, B. W. and Mooney, R. J. (eds.), Proceedings of the 7th International Machine Learning Conference, Austin, TX, 1990.Google Scholar
  31. De Raedt, L. and Bruynooghe, M. CLINT: A Multistrategy Interactive Concept Learner, in Machine Learning: A Multistrategy Approach, Vol. IV, R.S. Michalski and G. Tecuci (Eds.), Morgan Kaufmann, 1993 (to appear).Google Scholar
  32. Rumelhart, D. E., McClelland and the PDP Research Group, Parallel Distributed Processing, Vol, 1 & 2, A Bradford Book, The MIT Press, Cambridge, Massachusetts, 1986.Google Scholar
  33. Russell, S., The Use of Knowledge in Analogy and Induction, Morgan Kaufman Publishers, Inc., San Mateo, CA, 1989.zbMATHGoogle Scholar
  34. Schafer, D., (Ed.), Proceedings of the 3rd International Conference on Genetic Algorithms, George Mason University, June 4–7, 1989.Google Scholar
  35. Schum, D.,A., “Probability and the Processes of Discovery, Proof, and Choice,” Boston University Law Review, Vol. 66,No 3 and 4, May/July 1986.Google Scholar
  36. Segre, A. M. (Ed.), Proceedings of the Sixth International Workshop on Machine Learning, Cornell University, Ithaca, New York, June 26–27, 1989.Google Scholar
  37. Sutton, R. S. (Ed.), Special Issue on Reinforcement Learning, Machine Learning Journal, Vol. 8,No. 3/4, May 1992.Google Scholar
  38. Tecuci G., “A Multistrategy Learning Approach to Domain Modeling and Knowledge Acquisition,” in Kodratoff, Y., (ed.), Proceedings of the European Conference on Machine Learning, Porto, Springer-Verlag, 1991a.Google Scholar
  39. Tecuci G., “Steps Toward Automating Knowledge Acquisition for Expert Systems,” in Rappaport, A., Gaines, B., and Boose, J. (Eds.), Proceedings of the AAAI-91 Workshop on Knowledge Acquisition “From Science to Technology to Tools”, Anaheim, CA, July, 1991b.Google Scholar
  40. Tecuci, G. and Michalski, R.S., “A Method for Multistrategy Task-adaptive Learning Based on Plausible Justifications,” in Birnbaum, L., and Collins, G. (eds.) Machine Learning: Proceedings of the Eighth International Workshop, San Mateo, CA, Morgan Kaufmann, 1991a.Google Scholar
  41. Tecuci G., and Michalski R.S., Input “Understanding” as a Basis for Multistrategy Task-adaptive Learning, in Ras, Z., and Zemankova, M. (eds.), Proceedings of the 6th International Symposium on Methodologies for Intelligent Systems, Lecture Notes on Artificial Intelligence, Springer Verlag, 1991b.Google Scholar
  42. Touretzky, D., Hinton, G., and Sejnowski, T. (Eds.), Proceedings of the 1988 Connectionist Models, Summer School, Carnegie Mellon University, June 17–26, 1988.Google Scholar
  43. Utgoff, P. Shift of Bias for Inductive Concept Learning, in Machine Learning: An Artificial Intelligence Approach Vol. II, Michalski, R.S., Carbonell, J.G., and Mitchell, T. M. (Eds.), Morgan Kaufmann Publishers, 1986.Google Scholar
  44. Warmuth, M. & Valiant, L. (Eds.) (1991). Proceedings of the 4rd Annual Workshop on Computational Learning Theory, Santa Cruz, CA: Morgan Kaufmann.Google Scholar
  45. Whewell, W., History of the Inductive Sciences, 3 vols., Third edition, London, 1857.Google Scholar
  46. Wilkins, D.C., Clancey, W.J., and Buchanan, B.G., An Overview of the Odysseus Learning Apprentice, Kluwer Academic Press, New York, NY, 1986.Google Scholar
  47. Wnek, J., Sarma, J., Wahab, A. A. and Michalski, R.S., COMPARING LEARNING PARADIGMS VIA DIAGRAMMATIC VISUALIZATION: A Case Study in Concept Learning Using Symbolic, Neural Net and Genetic Algorithm Methods, Proceedings of the 5th International Symposium on Methodologies for Intelligent Systems, University of Tennessee, Knoxville, TN, North-Holland, October 24–27, 1990.Google Scholar
  48. Wnek, J. and Michalski, R.S., COMPARING SYMBOLIC AND SUBSYMBOLIC LEARNING: A Case Study, in Machine Learing: A Multistrategy Approach, Volume IV, R.S. Michalski and G. Tecuci (Eds.), Morgan Kaufmann, 1993.Google Scholar

Copyright information

© Kluwer Academic Publishers 1993

Authors and Affiliations

  • Ryszard S. Michalski
    • 1
  1. 1.Center for Artificial IntelligenceGeorge Mason UniversityFairfax

Personalised recommendations