Advertisement

There Is a Free Lunch for Hyper-Heuristics, Genetic Programming and Computer Scientists

  • Riccardo Poli
  • Mario Graff
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5481)

Abstract

In this paper we prove that in some practical situations, there is a free lunch for hyper-heuristics, i.e., for search algorithms that search the space of solvers, searchers, meta-heuristics and heuristics for problems. This has consequences for the use of genetic programming as a method to discover new search algorithms and, more generally, problem solvers. Furthermore, it has also rather important philosophical consequences in relation to the efforts of computer scientists to discover useful novel search algorithms.

Keywords

Search Space Search Algorithm Genetic Programming Evolutionary Computation Free Lunch 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bader-El-Den, M., Poli, R.: Generating SAT local-search heuristics using a GP hyper-heuristic framework. In: Proceedings of Evolution Artificielle (October 2007)Google Scholar
  2. 2.
    Bader-El-Den, M.B., Poli, R.: A GP-based hyper-heuristic framework for evolving 3-SAT heuristics. In: Thierens, D., Beyer, H.-G., Bongard, J., Branke, J., Clark, J.A., Cliff, D., Congdon, C.B., Deb, K., Doerr, B., Kovacs, T., Kumar, S., Miller, J.F., Moore, J., Neumann, F., Pelikan, M., Poli, R., Sastry, K., Stanley, K.O., Stutzle, T., Watson, R.A., Wegener, I. (eds.) GECCO 2007: Proceedings of the 9th annual conference on Genetic and evolutionary computation, vol. 2, p. 1749. ACM Press, New York (2007)Google Scholar
  3. 3.
    Banzhaf, W., Nordin, P., Keller, R.E., Francone, F.D.: Genetic Programming – An Introduction; On the Automatic Evolution of Computer Programs and its Applications. Morgan Kaufmann, San Francisco (1998)zbMATHGoogle Scholar
  4. 4.
    Burke, E., Kendall, G., Newall, J., Hart, E., Ross, P., Schulenburg, S.: Hyper-heuristics: an emerging direction in modern search technology. In: Glover, F.W., Kochenberger, G.A. (eds.) Handbook of metaheuristics, ch. 16, pp. 457–474. Kluwer Academic Publishers, Boston (2003)CrossRefGoogle Scholar
  5. 5.
    Burke, E.K., Hyde, M.R., Kendall, G.: Evolving bin packing heuristics with genetic programming. In: Runarsson, T.P., Beyer, H.-G., Burke, E.K., Merelo-Guervós, J.J., Whitley, L.D., Yao, X. (eds.) PPSN 2006. LNCS, vol. 4193, pp. 860–869. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  6. 6.
    Burke, E.K., Hyde, M.R., Kendall, G., Woodward, J.: Automatic heuristic generation with genetic programming: evolving a jack-of-all-trades or a master of one. In: Thierens, D., Beyer, H.-G., Bongard, J., Branke, J., Clark, J.A., Cliff, D., Congdon, C.B., Deb, K., Doerr, B., Kovacs, T., Kumar, S., Miller, J.F., Moore, J., Neumann, F., Pelikan, M., Poli, R., Sastry, K., Stanley, K.O., Stutzle, T., Watson, R.A., Wegener, I. (eds.) GECCO 2007: Proceedings of the 9th annual conference on Genetic and evolutionary computation, vol. 2, pp. 1559–1565. ACM Press, New York (2007)Google Scholar
  7. 7.
    Di Chio, C., Poli, R., Langdon, W.B.: Evolution of force-generating equations for PSO using GP. In: Manzoni, S., Palmonari, M., Sartori, F. (eds.) AI*IA Workshop on Evolutionary Computation, Evoluzionistico GSICE 2005, University of Milan Bicocca, Italy, September 20 (2005)Google Scholar
  8. 8.
    Fukunaga, A.: Automated discovery of composite SAT variable selection heuristics. In: Proceedings of the National Conference on Artificial Intelligence (AAAI), pp. 641–648 (2002)Google Scholar
  9. 9.
    Igel, C., Toussaint, M.: On classes of functions for which no free lunch results hold. Information Processing Letters 86(6), 317–321 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Igel, C., Toussaint, M.: A no-free-lunch theorem for non-uniform distributions of target functions. Journal of Mathematical Modelling and Algorithms 3, 313–322 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Keller, R.E., Poli, R.: Cost-benefit investigation of a genetic-programming hyperheuristic. In: Monmarché, N., Talbi, E.-G., Collet, P., Schoenauer, M., Lutton, E. (eds.) EA 2007. LNCS, vol. 4926, pp. 13–24. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  12. 12.
    Keller, R.E., Poli, R.: Linear genetic programming of metaheuristics. In: Thierens, D., Beyer, H.-G., Bongard, J., Branke, J., Clark, J.A., Cliff, D., Congdon, C.B., Deb, K., Doerr, B., Kovacs, T., Kumar, S., Miller, J.F., Moore, J., Neumann, F., Pelikan, M., Poli, R., Sastry, K., Stanley, K.O., Stutzle, T., Watson, R.A., Wegener, I. (eds.) GECCO 2007: Proceedings of the 9th annual conference on Genetic and evolutionary computation, p. 1753. ACM Press, New York (2007)Google Scholar
  13. 13.
    Keller, R.E., Poli, R.: Linear genetic programming of parsimonious metaheuristics. In: Proceedings of IEEE Congress on Evolutionary Computation (CEC) (September 2007)Google Scholar
  14. 14.
    Kibria, R.H., Li, Y.: Optimizing the initialization of dynamic decision heuristics in DPLL SAT solvers using genetic programming. In: Collet, P., Tomassini, M., Ebner, M., Gustafson, S., Ekárt, A. (eds.) EuroGP 2006. LNCS, vol. 3905, pp. 331–340. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  15. 15.
    Koza, J.R.: Genetic Programming: On the Programming of Computers by Natural Selection. MIT Press, Cambridge (1992)zbMATHGoogle Scholar
  16. 16.
    Oltean, M.: Evolving evolutionary algorithms for function optimization. In: Chen, K. (ed.) The 7th Joint Conference on Information Sciences, September 2003, vol. 1, pp. 295–298. Association for Intelligent Machinery (2003)Google Scholar
  17. 17.
    Oltean, M.: Evolving evolutionary algorithms using linear genetic programming. Evolutionary Computation 13(3), 387–410 (Fall 2005)CrossRefGoogle Scholar
  18. 18.
    Oltean, M., Dumitrescu, D.: Evolving TSP heuristics using multi expression programming. In: Bubak, M., van Albada, G.D., Sloot, P.M.A., Dongarra, J. (eds.) ICCS 2004. LNCS, vol. 3037, pp. 670–673. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  19. 19.
    Poli, R., Di Chio, C., Langdon, W.B.: Exploring extended particle swarms: a genetic programming approach. In: Beyer, H.-G., O’Reilly, U.-M., Arnold, D.V., Banzhaf, W., Blum, C., Bonabeau, E.W., Cantu-Paz, E., Dasgupta, D., Deb, K., Foster, J.A., de Jong, E.D., Lipson, H., Llora, X., Mancoridis, S., Pelikan, M., Raidl, G.R., Soule, T., Tyrrell, A.M., Watson, J.-P., Zitzler, E. (eds.) GECCO 2005: Proceedings of the 2005 conference on Genetic and evolutionary computation, vol. 1, pp. 169–176. ACM Press, New York (2005)Google Scholar
  20. 20.
    Poli, R., Langdon, W.B., Holland, O.: Extending particle swarm optimisation via genetic programming. In: Keijzer, M., Tettamanzi, A.G.B., Collet, P., van Hemert, J., Tomassini, M. (eds.) EuroGP 2005. LNCS, vol. 3447, pp. 291–300. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  21. 21.
    Poli, R., Langdon, W.B., McPhee, N.F.: A field guide to genetic programming (2008), http://lulu.com http://www.gp-field-guide.org.uk (With contributions by Koza, J.R.)
  22. 22.
    Poli, R., Woodward, J., Burke, E.K.: A histogram-matching approach to the evolution of bin-packing strategies. In: Proceedings of the IEEE Congress on Evolutionary Computation, Singapore (2007) (accepted)Google Scholar
  23. 23.
    Schumacher, C., Vose, M.D., Whitley, L.D.: The no free lunch and problem description length. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO), pp. 565–570. Morgan Kaufmann, San Francisco (2001)Google Scholar
  24. 24.
    Valsecchi, A., Vanneschi, L.: A study of some implications of the no free lunch theorem. In: Giacobini, M., Brabazon, A., Cagnoni, S., Di Caro, G.A., Drechsler, R., Ekárt, A., Esparcia-Alcázar, A.I., Farooq, M., Fink, A., McCormack, J., O’Neill, M., Romero, J., Rothlauf, F., Squillero, G., Uyar, A.Ş., Yang, S. (eds.) EvoWorkshops 2008. LNCS, vol. 4974, pp. 633–642. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  25. 25.
    Whitley, D., Watson, J.P.: Complexity theory and the no free lunch theorem. In: Burke, E.K., Kendall, G. (eds.) Search Methodologies: Introductory Tutorials in Optimization and Decision Support Techniques, ch. 11, pp. 317–339. Springer, US (2005)CrossRefGoogle Scholar
  26. 26.
    Wolpert, D., Macready, W.: No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation 1(1), 67–82 (1997)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Riccardo Poli
    • 1
  • Mario Graff
    • 1
  1. 1.School of Computer Science and Electronic EngineeringUniversity of EssexUK

Personalised recommendations