Prototype Based Supervised Concept Learning Using Genetic Algorithms

  • Sandip Sen
  • Leslie Knight
  • Kevin Legg


Prototypes have been proposed as representation of concepts that are used effectively by humans. Developing computational schemes for generating prototypes from examples, however, has proved to be a difficult problem. We present a novel genetic algorithm based prototype learning system, PLEASE, for constructing appropriate prototypes from classified training instances. After constructing a set of prototypes for each of the possible classes, the class of a new input instance is determined by the nearest prototype to this instance. Attributes are assumed to be ordinal in nature and prototypes are represented as sets of feature-value pairs. A genetic algorithm is used to evolve the number of prototypes per class and their positions on the input space. We present experimental results on a series of artificial problems of varying complexity. PLEASE performs competitively with several nearest neighbor classification algorithms and C4.5 on the problem set.We provide an analysis of the strengths and weaknesses of the current system.


Hepatitis Radar Brittleness Flare Posit 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [AKA91]
    David W. Aha, Dennis Kibler, and Marc K. Albert. Instance-based learning algorithms. Machine Learning, 6(1), 1991.Google Scholar
  2. [BFO84]
    L. Breiman, J.H. Friedman, and R.A. Olshen. Classification and Regression Trees. Wadsworth, Belmont, CA, 1984.MATHGoogle Scholar
  3. [BGH89]
    L.B. Booker, D.E. Goldberg, and J.H. Holland. Classifier systems and genetic algorithms. Artificial Intelligence, 40:235–282, 1989.CrossRefGoogle Scholar
  4. [Che92]
    K.J. Cherkauer. Genetic search for nearest-neighbor exemplars. In Proceedings of the Fourth Midwest Artificial Intelligence and Cognitive Science Society Conference, pages 87–91, 1992.Google Scholar
  5. [CKS+ 88]
    P. Cheeseman, J. Kelly, M. Self, J. Stutz, W. Taylor, and D. Freeman. Autoclass: A bayesian classification system. In Proceedings of the Fifth International Conference on Machine Learning, pages 54–64, 1988.Google Scholar
  6. [CS94]
    A.L. Corcoran and S. Sen. Using real-valued genetic algorithms to evolve rule sets for classification. In Proceedings of the IEEE Conference on Evolutionary Computation, pages 120–124, 1994.Google Scholar
  7. [DeJ90]
    Kenneth A. DeJong. Genetic-algorithm-based learning. In Y. Kodratoff and R.S. Michalski, editors, Machine Learning, Volume III. Morgan Kaufmann, Los Alamos, CA, 1990.Google Scholar
  8. [DSG93]
    Kenneth A. DeJong, William M. Spears, and Diana F. Gordon. Using genetic algorithms for concept learning. Machine Learning, 13:161–197, 1993.CrossRefGoogle Scholar
  9. [Fis87]
    Douglas H. Fisher. Knowledge acquisition via incremental conceptual clustering. Machine Learning, 2:139–172, 1987.Google Scholar
  10. [Go191]
    David Goldberg. Real-coded genetic algorithms, virtual alphabets, and blocking. Complex Systems, 5:139–168, 1991.MathSciNetMATHGoogle Scholar
  11. [GS93]
    David Perry Greene and Stephen F. Smith. Competition-based induction of decision models from examples. Machine Learning, 13:229–258, 1993.Google Scholar
  12. [HHNT86]
    John H. Holland, K.J. Holyoak, R.E. Nisbett, and P.R. Thagard. Induction: Processes of Inferences, Learning, and Discovery. MIT Press, Cambridge, MA, 1986.Google Scholar
  13. [Hin86]
    D.L. Hintzman. “schema abstraction” in a multiple trace memory model. Psychological Review, 93:411–428, 1986.CrossRefGoogle Scholar
  14. [Ho175]
    John H. Holland. Adpatation in natural and artificial systems. University of Michigan Press, Ann Arbor, MI, 1975.Google Scholar
  15. [Ho186]
    John H. Holland. Escaping brittleness: the possibilities of general-purpose learning algorithms applied to parallel rule-based systems. In R.S. Michalski, J.G. Carbonell, and T. M. Mitchell, editors, Machine Learning, an artificial intelligence approach: Volume II. Morgan Kaufmann, Los Alamos, CA, 1986.Google Scholar
  16. [JM91]
    Cezary Z. Janikow and Zbigniew Michalewicz. An experimental comparison of binary and floating point representations in genetic algorithms. In Proceedings of the 4th International Conference on Genetic Algorithms, pages 31–36, San Mateo, CA, 1991. Morgan Kaufman.Google Scholar
  17. [JM93]
    Cezary Z. Janikow and Zbigniew Michalewicz. A knowledge intensive genetic algorithm for supervised learning. Machine Learning, 13:198–228, 1993.CrossRefGoogle Scholar
  18. [KD91]
    James D. Kelly and Lawrence Davis. A hybrid genetic algorithm for classification. In Proceedings of the International Joint Conference on Artificial Intelligence, pages 645–650, 1991.Google Scholar
  19. [KM86]
    D. Kahneman and D.T. Miller. Norm theory: Comparing reality to its alternatives. Psychological Review, 93:136–153, 1986.CrossRefGoogle Scholar
  20. [Kru92]
    J.K. Kruschke. ALCOVE: An exemplar-based connectionist model of category learning. Psychological Review, 99:22–44, 1992.CrossRefGoogle Scholar
  21. [MA92]
    P.M. Murphy and D.W. Aha. UCI repository of machine learning databases [machine-readable data repository], 1992.Google Scholar
  22. [Mic83]
    R.S. Michalski. A theory and methodology of inductive learning. In R.S. Michalski, J. Carbonell, and T. Mitchell, editors, Machine Learning: An Artificial Intelligence Approach. Morgan Kaufman, Los Alamos, CA, 1983.Google Scholar
  23. [Mit86]
    T. Mitchell. Generalization as search. Artificial Intelligence, 18:203–226, 1986.CrossRefGoogle Scholar
  24. [Mla93]
    Dunja Mladenić. Combinatorial optimization in inductive concept learning. In Proceedings of the Tenth International Conference on Machine Learning, pages 205–211, 1993.Google Scholar
  25. [Qui86]
    Ross J. Quinlan. Induction of decision trees. Machine Learning, 1:81–106, 1986.Google Scholar
  26. [Qui90]
    Ross J. Quinlan. Learning logical definitions from relations. Machine Learning, 5(3):239–266, 1990.Google Scholar
  27. [Qui93]
    Ross J. Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo, California, 1993.Google Scholar
  28. [Ree72]
    S.K. Reed. Pattern recognition and categorization. Cognitive Psychology, 3:382–407, 1972.CrossRefGoogle Scholar
  29. [RHW86]
    D.E. Rumelhart, G.E. Hinton, and R.J. Williams. Learning internal representations by error propagation. In D.E. Rumelhart and J.L. McClelland, editors, Parallel Distributed Processing, volume 1. MIT Press, Cambridge, MA, 1986.Google Scholar
  30. [RZ85]
    David E. Rumelhart and David Zipser. Feature discovery by competitive learning. Cognitive Science, 9(1), 1985.Google Scholar
  31. [Sch78]
    D. L. Medin & M.M. Schaffer. Context theory of classification learning. Psychological Review, 85:207–238, 1978.CrossRefGoogle Scholar
  32. [She74]
    R.N. Shepard. Representation of structure in similarity data: Problems and prospects. Psychometrika, 39:373–421, 1974.MathSciNetMATHCrossRefGoogle Scholar
  33. [SK95]
    Sandip Sen and Leslie Knight. A genetic prototype learner. In Proceedings of the International Joint Conference on Artificial Intelligence, 1995.Google Scholar
  34. [Ska94]
    David B. Skalak. Prototype and feature selection by sampling and random mutation hill climbing algorithms. In Proceedings of the Eleventh International Conference on Machine Learning, pages 293–301, 1994.Google Scholar
  35. [SM81]
    E.E. Smith and D.L. Medin. Categories and concepts. Harvard University Press, Cambridge, MA, 1981.Google Scholar
  36. [Smi80]
    Steve F. Smith. A learning system based on genetic adaptive algorithms. PhD thesis, University of Pittsburgh, 1980. (Dissertation Abstracts Interna tional, 41, 4582B; University Microfilms No. 81–12638).Google Scholar
  37. [Smi89]
    Edward E. Smith. Concepts and induction. In Michael I. Posner, editor, Foundations of Cognitive Science. MIT Press, Cambridge, MA, 1989.Google Scholar
  38. [SMR84]
    E.E. Smith, D.L. Medin, and L.J. Rips. A psychological approach to concepts: Comments on rey’s “concepts and stereotypes” . Cognition, 17: 265— 274, 1984.Google Scholar
  39. [SW86]
    C. Stanfill and D. Waltz. Toward memory-based reasoning. Communications of the ACM, 29:1213–1228, 1986.CrossRefGoogle Scholar
  40. [V089]
    S. Vosniadou and A. Ortony. Similarity and Analogical Reasoning. Cambridge University Press, Cambridge, MA, 1989.Google Scholar
  41. [Whi89]
    D. Whitley. The genitor algorithm and selection pressure: Why rank-based allocation of reproductive trials is best. In Proceedings of the 3rd International Conference on Genetic Algorithms, pages 116–121, San Mateo, CA, 1989. Morgan Kaufman.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1997

Authors and Affiliations

  • Sandip Sen
    • 1
  • Leslie Knight
    • 1
  • Kevin Legg
    • 1
  1. 1.Dept of Mathematical & Computer SciencesThe University of TulsaTulsaUSA

Personalised recommendations