Advertisement

An Empirical Study of Functional Complexity as an Indicator of Overfitting in Genetic Programming

  • Leonardo Trujillo
  • Sara Silva
  • Pierrick Legrand
  • Leonardo Vanneschi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6621)

Abstract

Recently, it has been stated that the complexity of a solution is a good indicator of the amount of overfitting it incurs. However, measuring the complexity of a program, in Genetic Programming, is not a trivial task. In this paper, we study the functional complexity and how it relates with overfitting on symbolic regression problems. We consider two measures of complexity, Slope-based Functional Complexity, inspired by the concept of curvature, and Regularity-based Functional Complexity based on the concept of Hölderian regularity. In general, both complexity measures appear to be poor indicators of program overfitting. However, results suggest that Regularity-based Functional Complexity could provide a good indication of overfitting in extreme cases.

Keywords

Genetic Programming Complexity Measure Functional Complexity Toxicity Problem Complexity Score 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Jaffard, S.: Wavelet techniques in multifractal analysis. In: Fractal Geometry and Applications: A Jubilee of Benoit Mandelbrot, Proceedings of Symposium in Pure Mathematics, vol. 72, pp. 91–151 (2004)Google Scholar
  2. 2.
    Jones, T., Forrest, S.: Fitness distance correlation as a measure of problem difficulty for genetic algorithms. In: Proceedings of the Sixth International Conference on Genetic Algorithms, pp. 184–192. Morgan Kaufmann, San Francisco (1995)Google Scholar
  3. 3.
    Legrand, P., Lévy-Véhel, J.: Local regularity-based interpolation. In: WAVELET X, Part of SPIE’s Symposium on Optical Science and Technology, vol. 5207 (2003)Google Scholar
  4. 4.
    Lévy-Véhel, J., Legrand, P.: Signal and Image Processing with FRACLAB. In: Thinking in Patterns, pp. 321–322 (2004), http://fraclab.saclay.inria.fr/homepage.html
  5. 5.
    McDermott, J., Galvan-Lopez, E., O’Neill, M.: A fine-grained view of GP locality with binary decision diagrams as ant phenotypes. In: Schaefer, R., Cotta, C., Kołodziej, J., Rudolph, G. (eds.) PPSN XI. LNCS, vol. 6238, pp. 164–173. Springer, Heidelberg (2010)Google Scholar
  6. 6.
    Mitchell, T.: Machine Learning. McGraw-Hill, New York (1997)zbMATHGoogle Scholar
  7. 7.
    Morvan, J.-M.: Generalized Curvatures, 1st edn. Springer, Heidelberg (2008)CrossRefzbMATHGoogle Scholar
  8. 8.
    Rosca, J.: Generality versus size in genetic programming. In: Proceedings of Genetic Programming 1996, pp. 381–387. MIT Press, Cambridge (1996)Google Scholar
  9. 9.
    Silva, S., Almeida, J.: Gplab: A genetic programming toolbox for matlab. In: Proceedings of the Nordic MATLAB Conference, pp. 273–278 (2003)Google Scholar
  10. 10.
    Silva, S., Costa, E.: Dynamic limits for bloat control in genetic programming and a review of past and current bloat theories. Genetic Programming and Evolvable Machines 10(2), 141–179 (2009)CrossRefGoogle Scholar
  11. 11.
    Tricot, C.: Curves and Fractal Dimension. Springer, Heidelberg (1995)CrossRefzbMATHGoogle Scholar
  12. 12.
    Trujillo, L., Legrand, P., Lévy-Véhel, J.: The estimation of hölderian regularity using genetic programming. In: Proceedings of GECCO 2010, pp. 861–868. ACM, New York (2010)Google Scholar
  13. 13.
    Trujillo, L., Legrand, P., Olague, G., Pérez, C.: Optimization of the hölder image descriptor using a genetic algorithm. In: Proceedings of GECCO 2010, pp. 1147–1154. ACM, New York (2010)Google Scholar
  14. 14.
    Vanneschi, L., Castelli, M., Silva, S.: Measuring bloat, overfitting and functional complexity in genetic programming. In: Proceedings of GECCO 2010, pp. 877–884. ACM, New York (2010)Google Scholar
  15. 15.
    Vladislavleva, E.J., Smits, G.F., Den Hertog, D.: Order of nonlinearity as a complexity measure for models generated by symbolic regression via pareto genetic programming. IEEE Trans. Evol. Comp. 13, 333–349 (2009)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Leonardo Trujillo
    • 1
  • Sara Silva
    • 2
    • 3
  • Pierrick Legrand
    • 4
    • 5
  • Leonardo Vanneschi
    • 2
    • 6
  1. 1.Instituto Tecnológico de TijuanaTijuanaMéxico
  2. 2.KDBIO groupINESC-ID LisboaLisbonPortugal
  3. 3.CISUC, ECOS groupUniversity of CoimbraPortugal
  4. 4.IMB, Institut de Mathématiques de Bordeaux, UMR CNRS 5251France
  5. 5.INRIA Bordeaux Sud-OuestFrance
  6. 6.Department of Informatics, Systems and Communication (D.I.S.Co.)University of Milano-BicoccaMilanItaly

Personalised recommendations