Advertisement

Phase Transitions in Genetic Programming Search

  • Jason M. Daida
  • Ricky Tang
  • Michael E. Samples
  • Matthew J. Byom
Part of the Genetic and Evolutionary Computation book series (GEVO)

Abstract

Phase transitions occur in computational, as well as thermodynamic systems. Of particular interest is the possibility that phase transitions occur as a consequence of GP search. If this were so, it would allow for a statistical mechanics approach and quantitative comparisons of GP with a broad variety of rigorously described systems. This chapter summarizes our research group’s work in this area and describes a case study that illustrates what is involved in establishing the existence of phase transitions in GP search.

Keywords

Phase Transition Genetic Program Random Graph Critical Phenomenon Thermodynamic System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Achlioptas, D., Naor, A., and Peres, Y. (2005). Rigorous location of phase transitions in hard optimization problems. Nature, 435(7043):759–754.CrossRefGoogle Scholar
  2. Adamic, L.A., Lukose, R.M., Puniyani, A.R., and Huberman, B.A. (2001). Search in power-law networks. Phys Rev E, 64(4):046135/1–8.CrossRefGoogle Scholar
  3. Barabási, A.L. (2003). Emergence of scaling in complex networks. In Bornholdt, S. and Schuster, P., editors, Handbook of Graphs and Networks, pages 69–84. Wiley-VCH, Weinheim.Google Scholar
  4. Beottcher, S. and Percus, A.G. (2000). Combining local search with co-evolution in a remarkably simple way. In CEC, pages 1578–1584, La Jolla, CA, USA. IEEE.Google Scholar
  5. Beottcher, S. and Percus, A.G. (2001). External optimization for graph partioning. Phys Rev E, 64(2 II):26114/1–13.Google Scholar
  6. Bianconi, G. and Barabási, A.L. (2001). Bose-einstein condensation in complex networks. Physical Review Letters, 86(24):5632–5635.CrossRefGoogle Scholar
  7. Chan, M.H.W. (2001). Critical Phenomena. McGraw-Hill.Google Scholar
  8. Cheeseman, P., Kanefsky, B., and Taylor, W. (1991). Where the really hard problems are. In Proc lnt Joint Conf on AI, pages 331–337. Morgan Kaufmann.Google Scholar
  9. Clearwater, S.H. and Hogg, T. (1996). Problem structure heuristics and scaling behavior for genetic algorithms. AI, 81(1–2):327.MathSciNetGoogle Scholar
  10. Correale, L., Leone, M., Pagnani, A., Weight, M., and Zecchina, R (2006). Core percolation and onset of complexity in boolean networks. Phys Rev Letters, 96(1):018101/1–4.CrossRefGoogle Scholar
  11. Crutchfield, J.P. and Mitchell, M. (1995). The evolution of emergent computation. Proceedings of the National Academy of Sciences of the United States of America, 92(23): 10742–10746.MATHCrossRefGoogle Scholar
  12. Crutchfield, J.P. and Schuster, P. (2003). Evolutionary Dynamics: Exploring the Interplay of Selection, Accident, Neutrality, and function. Oxford University Press, New York.MATHGoogle Scholar
  13. Daida, Jason (2004). Considering the roles of structure in problem solving by a computer. In O’Reilly, Una-May, Yu, Tina, Riolo, Rick L., and Worzel, Bill, editors, Genetic Programming Theory and Practice II, chapter 5, pages 67–86. Springer, Ann Arbor.Google Scholar
  14. Daida, Jason, Ward, David, Hilss, Adam, Long, Stephen, and Hodges, Mark (2004). Visualizing the loss of diversity in genetic programming. In Proceedings of the 2004 IEEE Congress on Evolutionary Computation, pages 1225–1232, Portland, Oregon. IEEE Press.CrossRefGoogle Scholar
  15. Daida, Jason M. (2003). What makes a problem GP-hard? A look at how structure affects content. In Riolo, Rick L. and Worzel, Bill, editors, Genetic Programming Theory and Practice, chapter 7, pages 99–118. Kluwer.Google Scholar
  16. Daida, Jason M. (2005). Towards identifying populations that increase the likelihood of success in genetic programming. In Beyer, Hans-Georg, O’Reilly, Una-May, Arnold, Dirk V., Banzhaf, Wolfgang, Blum, Christian, Bonabeau, Eric W., Cantu-Paz, Erick, Dasgupta, Dipankar, Deb, Kalyanmoy, Foster, James A., de Jong, Edwin D., Lipson, Hod, Llora, Xavier, Mancoridis, Spiros, Pelikan, Martin, Raidl, Guenther R., Soule, Terence, Tyrrell, Andy M., Watson, Jean-Paul, and Zitzler, Eckart, editors, GECCO 2005: Proceedings of the 2005 conference on Genetic and evolutionary computation, volume 2, pages 1627–1634, Washington DC, USA. ACM Press.CrossRefGoogle Scholar
  17. Daida, Jason M., Bertram, Robert R., Polito 2, John A., and Stanhope, Stephen A. (1999a). Analysis of single-node (building) blocks in genetic programming. In Spector, Lee, Langdon, William B., O’Reilly, Una-May, and Angeline, Peter J., editors, Advances in Genetic Programming 3, chapter 10, pages 217–241. MIT Press, Cambridge, MA, USA.Google Scholar
  18. Daida, Jason M. and Hilss, Adam M. (2003). Identifying structural mechanisms in standard genetic programming. In Cantú-Paz, E., Foster, J. A., Deb, K., Davis, D., Roy, R., O’Reilly, U.-M., Beyer, H.-G., Standish, R., Kendall, G., Wilson, S., Harman, M., Wegener, J., Dasgupta, D., Potter, M. A., Schultz, A. C., Dowsland, K., Jonoska, N., and Miller, J., editors, Genetic and Evolutionary Computation — GECCO-2003, volume 2724 of LNCS, pages 1639–1651, Chicago. Springer-Verlag.Google Scholar
  19. Daida, Jason M., Hilss, Adam M., Ward, David J., and Long, Stephen L. (2003). Visualizing tree structures in genetic programming. In Cantú-Paz, E., Foster, J. A., Deb, K., Davis, D., Roy, R., O’Reilly, U.-M., Beyer, H.-G., Standish, R., Kendall, G., Wilson, S., Harman, M., Wegener, J., Dasgupta, D., Potter, M. A., Schultz, A. C., Dowsland, K., Jonoska, N., and Miller, J., editors, Genetic and Evolutionary Computation — GECCO-2003, volume 2724 of LNCS, pages 1652–1664, Chicago. Springer-Verlag.Google Scholar
  20. Daida, Jason M., Polito, John A., Stanhope, Steven A., Bertram, Robert R., Khoo, Jonathan C., and Chaudhary, Shahbaz A. (1999b). What makes a problem GP-hard? analysis of a tunably difficult problem in genetic programming. In Banzhaf, Wolfgang, Daida, Jason, Eiben, Agoston E., Garzon, Max H., Honavar, Vasant, Jakiela, Mark, and Smith, Robert E., editors, Proceedings of the Genetic and Evolutionary Computation Conference, volume 2, pages 982–989, Orlando, Florida, USA. Morgan Kaufmann.Google Scholar
  21. Daida, Jason M., Samples, Michael E., and Byom, Matthew J. (2005). Probing for limits to building block mixing with a tunably-difficult problem for genetic programming. In Beyer, Hans-Georg, O’Reilly, Una-May, Arnold, Dirk V, Banzhaf, Wolfgang, Blum, Christian, Bonabeau, Eric W., Cantu-Paz, Erick, Dasgupta, Dipankar, Deb, Kalyanmoy, Foster, James A., de Jong, Edwin D., Lipson, Hod, Llora, Xavier, Mancoridis, Spiros, Pelikan, Martin, Raidl, Guenther R., Soule, Terence, Tyrrell, Andy M., Watson, Jean-Paul, and Zitzler, Eckart, editors, GECCO 2005: Proceedings of the 2005 conference on Genetic and evolutionary computation, volume 2, pages 1713–1720, Washington DC, USA. ACM Press.CrossRefGoogle Scholar
  22. Daida, J.M., Bertram, R.R., Stanhope, S.A., Khoo, J.C., Chaudhary, S.A., Chaudhri, O.A., and II, J.A. Polito (2001). What makes a problem gp-hard? analysis of a tunably difficult problem in genetic programming. GPEM, 2(2):165–191.MATHGoogle Scholar
  23. Erdos, P. and Rényi, A. (1960). On the evolution of random graphs. Publications of the Mathematical Institute of the Hungarian Academy of Sciences, 5:17–61.Google Scholar
  24. Gao, Y. and Culberson, J. (2002). An analysis of phase transitions in nk landscapes. JAIR, 17:309–332.MATHMathSciNetGoogle Scholar
  25. Hartmann, A. and Weight, M. (2005). Phase transitions in combinatory optimization problems. Wiley-VCH, Berlin.Google Scholar
  26. Hogg, T., Huberman, B.A., and Williams, C.P. (1996). Phase transitions and the search problem. AI, 81(1–2):1–15.Google Scholar
  27. Huberman, B.A. and Hogg, T. (1987). Phase transitions and ai systems. AI, 33(2): 155–177.Google Scholar
  28. Huberman, B.A., Lukose, R.M., and Hogg, T. (1997). An economics approach to hard computational problems. Science, 275(5296):51–54.CrossRefGoogle Scholar
  29. Izumi, K. and Ueda, K. (2001). Phase transition in a foreign exchange market-analysis based on an artificial market approach. IEE TEC, 5(5):456.Google Scholar
  30. Janson, S., Luczak, T., and Rucinski, A. (1999). Random Graphs. John Wiley, New York.Google Scholar
  31. Koza, John R. (1992). Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge, MA, USA.MATHGoogle Scholar
  32. McPhee, Nicholas Freitag and Poli, Riccardo (2000). A schema theory analysis of the evolution of size in genetic programming with linear representations. Technical Report CSRP-00-22, University of Birmingham, School of Computer Science.Google Scholar
  33. Mitchell, D., Selman, B., and Levesque, H. (1992). Hard and easy distributions of sat problems. In Rosenbloom, P. and Szolovits, P., editors, Tenth National Conference on AI, pages 459–465, Menlo Park. AAAI.Google Scholar
  34. Mitchell, M., Hraber, P.T., and Crutchfield, J.P. (1993). Revisiting the edge of chaos: evolving cellular automata to perform computations. Complex Systems, 7(2):89–130.MATHGoogle Scholar
  35. Mohamed, H.M., Munzir, S., Abdulmuin, M.Z., and Hameida, S. (2000). Fuzzy modeling and control of a spark ignition engine idle mode. In TENCON, pages II-586–II-591, Kuala Lumpur. IEEE.Google Scholar
  36. Naudts, B. and Schoofs, L. (2002). Ga performance distributions and randomly generated binary constraint satisfaction problems. Theoretical COmputer Science, 287(1):167–185.MATHCrossRefMathSciNetGoogle Scholar
  37. Oates, M.J., Corne, D.W., and Loader, R.J. (2000). Tri-phase performance profile of evolutionary search on uni-and multi-modal search spaces. In CEC, pages 357–364, La Jolla. IEEE.Google Scholar
  38. Oda, A., Nagao, H., Kitagawa, Y., Shigeta, Y., and Yamaguchi, K. (2000). Theoretical studies on magnetic behavior in clusters by the genetic algorithms. International Journal of Quantum Chemistry, 80(4–5):646–656.CrossRefGoogle Scholar
  39. Samples, M.E., Byom, M.J., and Daida, J.M. (2006). Parameter sweeps for exploring parameter spaces of evolutionary algorithms. In Lobo, F.G., Lima, C.F., and Michalewicz, Z., editors, Parameter Settings in Evolutionary Algorithms, New York. Springer.Google Scholar
  40. Sellmyer, D.J. and Jaswal, S.S. (2002). Phase transitions. McGraw-Hill, New York.Google Scholar
  41. Shimazaki, H. and Niebur, E. (2005). Phase transitions in multiplicative competitive processes. Physical Review E, 72(1):1–4.CrossRefGoogle Scholar
  42. Stanley, H.E. (1971). Introduction to Phase Transitions and Critical Phenomena. Oxford University Press, New York.Google Scholar
  43. Toulouse, M., Crainic, T., and Sansó, B. (2004). Systemic behavior of cooperative search algorithms. Parallel Computing, 30(1):57–79.CrossRefMathSciNetGoogle Scholar
  44. van Hemert, J.I. and Urquhart, N.B. (2004). Phase transition properties of clustered traveling salesman problem instances generated with ec. In PPSN, pages 151–160, Birmingham. Springer-Verlag.Google Scholar
  45. Yamamoto, K. and Naito, S. (2002). A study on schema preservation by crossover. Systems and Computers in Japan, 33(2):64–76.CrossRefGoogle Scholar
  46. Zhang, F. and Dozier, G. (2004). A comparison of distributed restricted recombination operators for gec societies of hill-climbers: a disacsp perspective. In CEC, pages 1988–1995, Portland. IEEE.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  • Jason M. Daida
    • 1
  • Ricky Tang
    • 1
  • Michael E. Samples
    • 1
  • Matthew J. Byom
    • 1
  1. 1.Center for the Study of Complex Systems and Space Physics Research LaboratoryThe University of MichiganUSA

Personalised recommendations