Advertisement

Enhancing Efficiency of Hierarchical BOA Via Distance-Based Model Restrictions

  • Mark Hauschild
  • Martin Pelikan
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5199)

Abstract

This paper analyzes the effects of restricting probabilistic models in the hierarchical Bayesian optimization algorithm (hBOA) by defining a distance metric over variables and disallowing dependencies between variables at distances greater than a given threshold. We argue that by using prior problem-specific knowledge, it is often possible to develop a distance metric that closely corresponds to the strength of interactions between variables. This distance metric can then be used to speed up model building in hBOA. Three test problems are considered: 3D Ising spin glasses, random additively decomposable problems, and the minimum vertex cover.

Keywords

Bayesian Network Spin Glass Distance Restriction Distribution Algorithm Random Instance 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Pelikan, M., Goldberg, D.E.: Escaping hierarchical traps with competent genetic algorithms. In: Genetic and Evolutionary Computation Conf (GECCO 2001), pp. 511–518 (2001)Google Scholar
  2. 2.
    Pelikan, M.: Hierarchical Bayesian optimization algorithm: Toward a new generation of evolutionary algorithms. Springer, Heidelberg (2005)Google Scholar
  3. 3.
    Sastry, K., Goldberg, D.E., Pelikan, M.: Don’t evaluate, inherit. In: Genetic and Evolutionary Computation Conf .(GECCO 2001), pp. 551–558 (2001)Google Scholar
  4. 4.
    Pelikan, M., Sastry, K.: Fitness inheritance in the Bayesian optimization algorithm. In: Genetic and Evolutionary Computation Conf (GECCO 2004), vol. 2, pp. 48–59 (2004)Google Scholar
  5. 5.
    Lima, C.F., Pelikan, M., Sastry, K., Butz, M.V., Goldberg, D.E., Lobo, F.G.: Substructural neighborhoods for local search in the Bayesian optimization algorithm. Parallel Problem Solving from Nature, 232–241 (2006)Google Scholar
  6. 6.
    Pelikan, M., Sastry, K., Goldberg, D.E.: Sporadic model building for efficiency enhancement of hierarchical BOA. In: Genetic and Evolutionary Computation Conf (GECCO 2006), pp. 405–412 (2006)Google Scholar
  7. 7.
    Sastry, K., Pelikan, M., Goldberg, D.E.: Efficiency enhancement of estimation of distribution algorithms. In: Pelikan, M., Sastry, K., Cantú-Paz, E. (eds.) Scalable Optimization via Probabilistic Modeling: From Algorithms to Applications, pp. 161–185. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  8. 8.
    Baluja, S.: Population-based incremental learning: A method for integrating genetic search based function optimization and competitive learning. Tech. Rep. No. CMU-CS-94-163, Carnegie Mellon University, Pittsburgh, PA (1994)Google Scholar
  9. 9.
    Mühlenbein, H., Paaß, G.: From recombination of genes to the estimation of distributions I. Binary parameters. Parallel Problem Solving from Nature, 178–187 (1996)Google Scholar
  10. 10.
    Larranaga, P., Lozano, J.A. (eds.): Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation. Springer, Heidelberg (2001)Google Scholar
  11. 11.
    Schwarz, J., Ocenasek, J.: A problem-knowledge based evolutionary algorithm KBOA for hypergraph partitioning, Personal communication (2000)Google Scholar
  12. 12.
    Mühlenbein, H., Mahnig, T.: Evolutionary optimization and the estimation of search distributions with applications to graph bipartitioning. International Journal on Approximate Reasoning 31(3), 157–192 (2002)CrossRefzbMATHGoogle Scholar
  13. 13.
    Baluja, S.: Incorporating a priori knowledge in probabilistic-model based optimization. In: Pelikan, M., Sastry, K., Cantú-Paz, E. (eds.) Scalable optimization via probabilistic modeling: From algorithms to applications, pp. 205–219. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  14. 14.
    Santana, R., Larrañaga, P., Lozano, J.A.: The role of a priori information in the minimization of contact potentials by means of estimation of distribution algorithms. In: Marchiori, E., Moore, J.H., Rajapakse, J.C. (eds.) EvoBIO 2007. LNCS, vol. 4447, pp. 247–257. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  15. 15.
    Hauschild, M., Pelikan, M., Sastry, K., Goldberg, D.E.: Analyzing probabilistic models in hierarchical boa on traps and spin glasses. In: Genetic and Evolutionary Computation Conference (GECCO 2008), vol. I, pp. 523–530 (2008)Google Scholar
  16. 16.
    Larrañaga, P., Lozano, J.A. (eds.): Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation. Kluwer, Boston (2002)Google Scholar
  17. 17.
    Pelikan, M., Goldberg, D.E., Lobo, F.: A survey of optimization by building and using probabilistic models. Computational Optimization and Applications 21(1), 5–20 (2002)CrossRefzbMATHMathSciNetGoogle Scholar
  18. 18.
    Harik, G.R.: Finding multimodal solutions using restricted tournament selection. In: International Conference on Genetic Algorithms (ICGA 1995), pp. 24–31 (1995)Google Scholar
  19. 19.
    Howard, R.A., Matheson, J.E.: Influence diagrams. In: Howard, R.A., Matheson, J.E. (eds.) Readings on the principles and applications of decision analysis, vol. II, pp. 721–762. Strategic Decisions Group, Menlo Park (1981)Google Scholar
  20. 20.
    Pearl, J.: Probabilistic reasoning in intelligent systems: Networks of plausible inference. Morgan Kaufmann, San Mateo (1988)Google Scholar
  21. 21.
    Chickering, D.M., Heckerman, D., Meek, C.: A Bayesian approach to learning Bayesian networks with local structure. Technical Report MSR-TR-97-07, Microsoft Research, Redmond, WA (1997)Google Scholar
  22. 22.
    Friedman, N., Goldszmidt, M.: Learning Bayesian networks with local structure. In: Jordan, M.I. (ed.) Graphical models, pp. 421–459. MIT Press, Cambridge (1999)Google Scholar
  23. 23.
    Pelikan, M., Kalapala, R., Hartmann, A.K.: Hybrid evolutionary algorithms on minimum vertex cover for random graphs. In: Genetic and Evolutionary Computation Conference (GECCO 2007), pp. 547–554 (2007)Google Scholar
  24. 24.
    Hauschild, M., Pelikan, M., Lima, C., Sastry, K.: Analyzing probabilistic models in hierarchical boa on traps and spin glasses. In: Genetic and Evolutionary Computation Conference (GECCO 2007), vol. I, pp. 523–530 (2007)Google Scholar
  25. 25.
    Barahona, F.: On the computational complexity of Ising spin glass models. Journal of Physics A: Mathematical, Nuclear and General 15(10), 3241–3253 (1982)CrossRefMathSciNetGoogle Scholar
  26. 26.
    Pelikan, M., Sastry, K., Butz, M.V., Goldberg, D.E.: Performance of evolutionary algorithms on random decomposable problems. Parallel Problem Solving from Nature (PPSN IX), 788–797 (2006)Google Scholar
  27. 27.
    Sastry, K., Pelikan, M., Goldberg, D.E.: Empirical analysis of ideal recombination on random decomposable problems. In: Genetic and Evolutionary Computation Conference (GECCO 2007), pp. 1388–1395 (2007)Google Scholar
  28. 28.
    Pelikan, M., Sastry, K., Butz, M.V., Goldberg, D.E.: Hierarchical BOA on random decomposable problems. MEDAL Report No. 2006001, Missouri Estimation of Distribution Algorithms Laboratory, Univ. of Missouri–St. Louis (2006)Google Scholar
  29. 29.
    Karp, R.M.: Reducibility among combinatorial problems. In: Miller, R.E., Thatcher, J.W. (eds.) Complexity of Computer Computations, pp. 85–103. Plenum Press (1972)Google Scholar
  30. 30.
    Bollobas, B.: Random Graphs. Cambridge University Press, Cambridge (2001)CrossRefzbMATHGoogle Scholar
  31. 31.
    Weigt, M., Hartmann, A.K.: Minimal vertex covers on finite-connectivity random graphs - a hard-sphere lattice-gas picture. Physical Review E 63, 056127 (2001)CrossRefGoogle Scholar
  32. 32.
    Sastry, K.: Evaluation-relaxation schemes for genetic and evolutionary algorithms. Master’s thesis, University of Illinois at Urbana-Champaign, Department of General Engineering, Urbana, IL (2001)Google Scholar
  33. 33.
    Pelikan, M., Sastry, K., Goldberg, D.E.: Scalability of the Bayesian optimization algorithm. International Journal of Approximate Reasoning 31(3), 221–258 (2002)CrossRefzbMATHMathSciNetGoogle Scholar
  34. 34.
    Goldberg, D.E.: The design of innovation: Lessons from and for competent genetic algorithms. Kluwer, Dordrecht (2002)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Mark Hauschild
    • 1
  • Martin Pelikan
    • 1
  1. 1.Missouri Estimation of Distribution Algorithms Laboratory, 320 CCBUniversity of Missouri in St. Louis; One University Blvd.St. LouisUSA

Personalised recommendations