A Quantitative Study of Learning and Generalization in Genetic Programming

  • Mauro Castelli
  • Luca Manzoni
  • Sara Silva
  • Leonardo Vanneschi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6621)


The relationship between generalization and solutions functional complexity in genetic programming (GP) has been recently investigated. Three main contributions are contained in this paper: (1) a new measure of functional complexity for GP solutions, called Graph Based Complexity (GBC) is defined and we show that it has a higher correlation with GP performance on out-of-sample data than another complexity measure introduced in a recent publication. (2) A new measure is presented, called Graph Based Learning Ability (GBLA). It is inspired by the GBC and its goal is to quantify the ability of GP to learn “difficult” training points; we show that GBLA is negatively correlated with the performance of GP on out-of-sample data. (3) Finally, we use the ideas that have inspired the definition of GBC and GBLA to define a new fitness function, whose suitability is empirically demonstrated. The experimental results reported in this paper have been obtained using three real-life multidimensional regression problems.


Root Mean Square Error Cross Correlation Genetic Programming Complexity Measure Minimum Description Length 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Amil, N.M., Bredeche, N., Gagné, C., Gelly, S., Schoenauer, M., Teytaud, O.: A statistical learning perspective of genetic programming. In: Vanneschi, L., Gustafson, S., Moraglio, A., De Falco, I., Ebner, M. (eds.) EuroGP 2009. LNCS, vol. 5481, pp. 327–338. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  2. 2.
    Archetti, F., Lanzeni, S., Messina, E., Vanneschi, L.: Genetic programming for human oral bioavailability of drugs. In: Keijzer, M., et al. (eds.) GECCO 2006, vol. 1, pp. 255–262. ACM Press, New York (2006)Google Scholar
  3. 3.
    Banzhaf, W., Francone, F.D., Nordin, P.: The effect of extensive use of the mutation operator on generalization in genetic programming using sparse data sets. In: Ebeling, W., Rechenberg, I., Voigt, H.-M., Schwefel, H.-P. (eds.) PPSN 1996. LNCS, vol. 1141, pp. 300–309. Springer, Heidelberg (1996)CrossRefGoogle Scholar
  4. 4.
    Duda, R.O., Hart, P.E.: Pattern Classification and Scene Analysis. Wiley, Chichester (1973)zbMATHGoogle Scholar
  5. 5.
    Francone, F.D., Nordin, P., Banzhaf, W.: Benchmarking the generalization capabilities of a compiling genetic programming system using sparse data sets. In: Koza, J.R., et al. (eds.) GP 1996, pp. 72–80. MIT Press, Cambridge (1996)Google Scholar
  6. 6.
    Gagné, C., Schoenauer, M., Parizeau, M., Tomassini, M.: Genetic programming, validation sets, and parsimony pressure. In: Collet, P., Tomassini, M., Ebner, M., Gustafson, S., Ekárt, A. (eds.) EuroGP 2006. LNCS, vol. 3905, pp. 109–120. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  7. 7.
    Gomez, F.J., Togelius, J., Schmidhuber, J.: Measuring and optimizing behavioral complexity for evolutionary reinforcement learning. In: Alippi, C., Polycarpou, M., Panayiotou, C., Ellinas, G. (eds.) ICANN 2009. LNCS, vol. 5769, pp. 765–774. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  8. 8.
    Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge (1992)zbMATHGoogle Scholar
  9. 9.
    Kushchu, I.: An evaluation of evolutionary generalization in genetic programming. Artificial Intelligence Review 18(1), 3–14 (2002)CrossRefzbMATHGoogle Scholar
  10. 10.
    Langdon, W.B., Poli, R.: Foundations of Genetic Programming. Springer, Heidelberg (2002)CrossRefzbMATHGoogle Scholar
  11. 11.
    O’Neill, M., Vanneschi, L., Gustafson, S., Banzhaf, W.: Open issues in genetic programming. Genetic Programming and Evolvable Machines 11(3-4), 339–363 (2010)CrossRefGoogle Scholar
  12. 12.
    Poli, R., Langdon, W.B., McPhee, N.F.: A field guide to genetic programming (2008), Published via and freely available at
  13. 13.
    Rissanen, J.: Modeling by shortest data description. Automatica 14, 465–471 (1978)CrossRefzbMATHGoogle Scholar
  14. 14.
    Rosca, J.: Generality versus size in genetic programming. In: Koza, J.R., et al. (eds.) GP 1996, pp. 381–387. MIT Press, Cambridge (1996)Google Scholar
  15. 15.
    Silva, S., Costa, E.: Dynamic limits for bloat control in genetic programming and a review of past and current bloat theories. Genetic Programming and Evolvable Machines 10(2), 141–179 (2009)CrossRefGoogle Scholar
  16. 16.
    Silva, S., Vanneschi, L.: Operator equalisation, bloat and overfitting: a study on human oral bioavailability prediction. In: Raidl, G., et al. (eds.) GECCO 2009, pp. 1115–1122. ACM, New York (2009)Google Scholar
  17. 17.
    Vanneschi, L., Castelli, M., Silva, S.: Measuring bloat, overfitting and functional complexity in genetic programming. In: GECCO 2010, pp. 877–884. ACM, New York (2010)Google Scholar
  18. 18.
    Vanneschi, L., Silva, S.: Using operator equalisation for prediction of drug toxicity with genetic programming. In: Lopes, L.S., Lau, N., Mariano, P., Rocha, L.M. (eds.) EPIA 2009. LNCS, vol. 5816, pp. 65–76. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  19. 19.
    Vladislavleva, E.J., Smits, G.F., den Hertog, D.: Order of nonlinearity as a complexity measure for models generated by symbolic regression via pareto genetic programming. IEEE Transactions on Evolutionary Computation 13(2), 333–349 (2009)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Mauro Castelli
    • 1
  • Luca Manzoni
    • 1
  • Sara Silva
    • 2
  • Leonardo Vanneschi
    • 1
    • 2
  1. 1.Dipartimento di Informatica, Sistemistica e Comunicazione (D.I.S.Co.)University of Milano-BicoccaMilanItaly
  2. 2.KDBIO groupINESC-ID LisboaLisbonPortugal

Personalised recommendations