Mining Evolving Learning Algorithms

  • András Joó
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5481)


This paper presents an empirical method to identify salient patterns in tree based Genetic Programming. By using an algorithm derived from tree mining techniques and measuring the destructiveness of replacing patterns, we are able to identify those patterns that are responsible for the increased fitness of good individuals. The method is demonstraded on the evolution of learning rules for binary perceptrons.


Genetic programming perceptron learning rule tree mining 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Langdon, W., Poli, R.: Foundations of Genetic Programming. Springer, Heidelberg (2002)CrossRefzbMATHGoogle Scholar
  2. 2.
    Rattray, M., Saad, D.: Globally optimal on-line learning rules for multi-layer neural networks. Journal of Physics A: Mathematical and General 30(22), L771–L776 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Chalmers, D.J.: The evolution of learning: An experiment in genetic connectionism. In: Touretsky, D.S., Elman, J.L., Sejnowski, T.J., Hinton, G.E. (eds.) Proceedings of the 1990 Connectionist Summer School, pp. 81–90. Morgan Kaufmann, San Francisco (1990)Google Scholar
  4. 4.
    Baxter, J.: The evolution of learning algorithms for artificial neural networks. Complex Systems, 313–326 (1992)Google Scholar
  5. 5.
    Bengio, S., Bengio, Y., Cloutier, J.: Use of genetic programming for the search of a new learning rule for neutral networks. In: Proceedings of the 1994 IEEE World Congress on Computational Intelligence, Orlando, Florida, USA, vol. 1, pp. 324–327. IEEE Press, Los Alamitos (1994)Google Scholar
  6. 6.
    Radi, A., Poli, R.: Genetic programming can discover fast and general learning rules for neural networks. In: Genetic Programming 1998: Proceedings of the Third Annual Conference, University of Wisconsin, Madison, Wisconsin, USA, pp. 314–323. Morgan Kaufmann, San Francisco (1998)Google Scholar
  7. 7.
    Radi, A., Poli, R.: Discovery of backpropagation learning rules using genetic programming. In: Proceedings of the 1998 IEEE World Congress on Computational Intelligence, Anchorage, vol. 1, pp. 371–375. IEEE Press, Los Alamitos (1998)Google Scholar
  8. 8.
    Radi, A., Poli, R.: Discovering efficient learning rules for feedforward neural networks using genetic programming. Technical Report CSM-360, Department of Computer Science, University of Essex, Colchester, UK (January 2002)Google Scholar
  9. 9.
    Neirotti, J.P., Caticha, N.: Dynamics of the evolution of learning algorithms by selection. Physical Review E 67, 041912 (2003)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Tackett, W.A.: Mining the genetic program. IEEE Expert 10(3), 28–38 (1995)CrossRefGoogle Scholar
  11. 11.
    Smart, W., Andreae, P., Zhang, M.: Empirical analysis of GP tree-fragments. In: Ebner, M., O’Neill, M., Ekárt, A., Vanneschi, L., Esparcia-Alcázar, A.I. (eds.) EuroGP 2007. LNCS, vol. 4445, pp. 55–67. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  12. 12.
    Smart, W., Zhang, M.: Empirical analysis of schemata in genetic programming using maximal schemata and MSG. In: 2008 IEEE World Congress on Computational Intelligence, Hong Kong. IEEE Computational Intelligence Society, IEEE Press, Los Alamitos (2008)Google Scholar
  13. 13.
    McKay, R., Shin, J., Hoang, T.H., Nguyen, X.H., Mori, N.: Using compression to understand the distribution of building blocks in genetic programming populations. IEEE Congress on Evolutionary Computation, 2501–2508 (2007)Google Scholar
  14. 14.
    Nowostawski, M., Poli, R.: Parallel genetic algorithm taxonomy. In: Nowostawski, M., Poli, R. (eds.) Third International Conference on Knowledge-Based Intelligent Information Engineering Systems, December 1999, pp. 88–92 (1999)Google Scholar
  15. 15.
    Cantu-Paz, E.: Efficient and Accurate Parallel Genetic Algorithms (Genetic Algorithms and Evolutionary Computation 1), 1st edn. Springer, Heidelberg (2000)zbMATHGoogle Scholar
  16. 16.
    Poli, R., Langdon, W.B.: A new schema theory for genetic programming with one-point crossover and point mutation. In: Genetic Programming 1997: Proceedings of the Second Annual Conference, pp. 278–285. Morgan Kaufmann, San Francisco (1997)Google Scholar
  17. 17.
    Mori, N., McKay, R., Nguyen, X.H., Essam, D.: Equivalent decision simplification: A new method for simplifying algebraic expressions in genetic programming. In: Proceedings of 11th Asia-Pacific Workshop on Intelligent and Evolutionary Systems (2007)Google Scholar
  18. 18.
    Chi, Y., Muntz, R.R., Nijssen, S., Kok, J.N.: Frequent subtree mining - an overview. Fundamenta Informaticae 66(1-2), 161–198 (2004)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • András Joó
    • 1
  1. 1.NCRGAston UniversityBirminghamUK

Personalised recommendations