Skip to main content
SpringerLink
Log in

Opportunities for Expensive Optimization with Estimation of Distribution Algorithms

  • Chapter
Computational Intelligence in Expensive Optimization Problems

Part of the book series: Adaptation Learning and Optimization ((ALO,volume 2))

Abstract

The chapter claims that the search distributions of Estimation of Distribution Algorithms (EDAs) contain much information that can be obtained with the help of modern statistical techniques to create powerful strategies for expensive optimization. For example, it shows how the regularization of some parameters of the EDAs probabilistic models can yield dramatic improvements in efficiency. In this context a new class, Shrinkage EDAs, based on shrinkage estimation is presented. Also, a novelmutation operator based on a regularization of the entropy is discussed. Another key contribution of the chapter is the development of a new surrogate fitness model based on the search distributions. With this method the evolution starts in the fitness landscape, switches to the log-probability landscape of the model and then backtracks to continue in the original landscape if the optimum is not found. For the sake of completeness the chapter reviews other techniques for improving the sampling efficiency of EDAs. The theoretical presentation is accompanied by numerical simulations that support the main claims of the chapter.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 259.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 329.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 329.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Beinlich, I., Chavez, H.R., Cooper, G.: The ALARM Monitoring System: a Case Study with two Probabilistic Inference Techniques for Belief Networks. In: Artificial Intelligence in Medical Care, pp. 247–256 (1989)

    Google Scholar 

  2. Blanco, R., Lozano, J.A.: Empirical comparison of Estimation of Distribution Algorithms in combinatorial optimization. In: Larrañaga, P., Lozano, J.A. (eds.) Estimation of Distribution Algorithms. A New Tool for Evolutionar y Computation, Kluwer Academic Publishers, Dordrecht (2002)

    Google Scholar 

  3. Brown, L.E., Tsamardinos, I., Aliferis, C.F.: A comparison of novel and state-of-the-art polynomial bayesian network learning algorithms. In: AAAI, pp. 739–745 (2005)

    Google Scholar 

  4. Chiba, K., Jeong, S., Shigeru, O., Morino, H.: Data mining for multidisciplinary design space of regional-jet wing. In: Proceedings the 2005 IEEE Congress on Evolutionary Computation CEC 2005, pp. 2333–23340 (2005)

    Google Scholar 

  5. Deb, K., Goldberg, D.E.: Analyzing deception in trap functions. In: Analyzing deception in trap functions. Foundations of Genetic Algorithms, vol. 2, pp. 93–108 (1993)

    Google Scholar 

  6. Eby, D., Averill, R.C., Punch, W.F.I., Goodman, E.D.: Evaluation of injection island ga performance on flywheel design optimization. In: Proceedings of the Third Conference on Adaptive Computing in Design and Manufacturing (1998)

    Google Scholar 

  7. Groppo, T.: Learning bayesian networks skeleton: A comparison between TPDA and PMMS algorithm. Master’s thesis, Universite Claude Bernard Lyon I (2006)

    Google Scholar 

  8. Ireland, C.T., Kullback, S.: Contingency Tables with Given Marginals. Biometrika 55, 179–188 (1968)

    Article  MATH  MathSciNet  Google Scholar 

  9. Jaynes, E.T.: Information Theory and Statistical Mechanics. Physics Review 6, 620–643 (1957)

    Article  MathSciNet  Google Scholar 

  10. Jiroušek, R., Přeučil, S.: On the Effective Implementation of the Iterative Proportional Fitting Procedure. Comput. Statistics and Data Analysis 19, 177–189 (1995)

    Article  MATH  Google Scholar 

  11. Kim, H.S., Cho, S.B.: An efficient genetic algorithm with less fitness evaluation by clustering. In: Proceedings of 2001 IEEE Conference on Evolutionary Computation, pp. 887–894 (2001)

    Google Scholar 

  12. Larrañaga, P., Lozano, J.A. (eds.): Estimation of Distribution Algorithms. A New Tool for Evolutionary Computation. Kluwer Academic Publishers, Dordrecht (2002)

    MATH  Google Scholar 

  13. Larrañaga, P., Lozano, J.A., Miqueles, T., E-Bengoetxea: Experimental Results in Function Optimization with EDAs in Continuos Domain. In: Larrañaga, P., Lozano, J.A. (eds.) Estimation of Distribution Algorithms. A New Tool for Evolutionary Computation. Kluwer Academic Publishers, Dordrecht (2002)

    Google Scholar 

  14. Lauritzen, S.L.: Graphical Models. Oxford Press (1996)

    Google Scholar 

  15. Ledoit, O., Wolf, M.: Improved estimation of the covariance matrix of stock returns with an application to portfolio selection. J. Empir. Finance 10, 603–621 (2003)

    Article  Google Scholar 

  16. Lewis, P.M.: Approximating Probability Distributions to Reduce Storage Requirements. Information and Control 2, 214–225 (1959)

    Article  MATH  MathSciNet  Google Scholar 

  17. Liang, K.H., Yao, X., Newton, C.: Evolutionary search of approximated n-dimensional landscapes. International Journal of Knowledge-Based Intelligent Engineering Systems 4(3), 172–183 (2000)

    Google Scholar 

  18. Lozano, J., Larrañaga, P., Inza, I., Bengoetxea, E. (eds.): Towards a new Evolutionary Computation. Advances on Estimation of Distribution Algorithms. Studies in Fuzziness and Soft Computing. Springer, Heidelberg (2006)

    MATH  Google Scholar 

  19. Madera, J.: Hacia una Generación Eficiente de Algoritmos Evolutivos con Estimación de Distribuciones: Pruebas de (In)dependencia +Paralelismo. PhD thesis, Instituto de Cibernética, Matemática y Física, La Habana (in Spanish). Adviser: A. Ochoa (2009)

    Google Scholar 

  20. Madera, J., Ochoa, A.: Un EDA basado en aprendizaje Max-Min con escalador de colinas. Tech. Rep. 396, ICIMAF (2006)

    Google Scholar 

  21. Madera, J., Ochoa, A.: Algoritmos Evolutivos con Estimación de Distribuciones Bayesianas basados en pruebas de (in)dependencia. Tech. Rep. 474, ICIMAF (2008)

    Google Scholar 

  22. Mahnig, T., Mühlenbein, H.: Comparing the Adaptive Boltzmann Selection Schedule SDS to Truncation Selection. In: Third International Symposium on Adaptive Systems, ISAS 2001, Evolutionary Computation and Probabilistic Graphical Models, La Habana, pp. 121–128 (2001)

    Google Scholar 

  23. Meyer, C.H.: Korrektes Schlies̈en bei Unvollständiger Information. PhD thesis, Fernuniversität Hagen (1998) (in German)

    Google Scholar 

  24. Mühlenbein, H., Mahnig, T.: Mathematical analysis of evolutionary algorithms for optimization. In: Proceedings of the Third International Symposium on Adaptive Systems, pp. 166–185 (2001)

    Google Scholar 

  25. Mühlenbein, H., Mahnig, T.: Evolutionary Optimization and the Estimation of Search Distributions. Journal of Approximate Reasoning 31(3), 157–192 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  26. Mühlenbein, H., Paas, G.: From Recombination of Genes to the Estimation of Distributions I. Binary Parameters. In: Ebeling, W., Rechenberg, I., Voigt, H.-M., Schwefel, H.-P. (eds.) PPSN 1996. LNCS, vol. 1141, pp. 178–187. Springer, Heidelberg (1996)

    Chapter  Google Scholar 

  27. Mühlenbein, H., Mahnig, T., Ochoa, A.: Schemata, Distributions and Graphical Models in Evolutionary Optimization. Journal of Heuristics 5(2), 213–247 (1999)

    Article  Google Scholar 

  28. Nilsson, D.: An Efficient Algorithm for Finding the M most Probable Configuration in Bayesian Networks. Statistics and Computing 2, 159–173 (1998)

    Article  Google Scholar 

  29. Ochoa, A.: Linear Entropic Mutation (submitted for publication) (2009)

    Google Scholar 

  30. Ochoa, A.: The Random Class of B-functions (unpublished) (2009)

    Google Scholar 

  31. Ochoa, A., Soto, M.: Partial Evaluation in Genetic Algorithms. In: IEA/AIE 1997: Proceedings of the 10th International Conference on Industrial and Engineering Applications of Artificial Intelligence and Expert Systems, pp. 217–222. Goose Pond Press, Atlanta (1997)

    Google Scholar 

  32. Ochoa, A., Soto, M.: Linking Entropy to Estimation of Distribution Algorithms. In: Lozano, J., Larrañaga, P., Inza, I., Bengoetxea, E. (eds.) Towards a new Evolutionary Computation. Advances on Estimation of Distribution Algorithms. Studies in Fuzziness and Soft Computing, pp. 1–38. Springer, Heidelberg (2006)

    Google Scholar 

  33. Ochoa, A., Soto, M.: On the Performance of the Bayesian Optimization Algorithm with B-functions. Tech. Rep. 383, ICIMAF (2006)

    Google Scholar 

  34. Ochoa, A., Höns, R., Soto, M., Müehlenbein, H.: A Maximum Entropy Approach to Sampling in EDA-the Single Connected case. In: Sanfeliu, A., Ruiz-Shulcloper, J. (eds.) CIARP 2003. LNCS, vol. 2905, pp. 683–690. Springer, Heidelberg (2003)

    Google Scholar 

  35. Ong, Y.S., Nair, P.B., Keane, A.J.: Evolutionary optimization of computationally expensive problems via surrogate modeling. American Institute of Aeronautics and Astronautics Journal 41(4), 687–696 (2003)

    Google Scholar 

  36. Pelikan, M., Goldberg, D.E., Cantú-Paz, E.: BOA: The Bayesian optimization algorithm. In: Banzhaf, W., Daida, J., Eiben, A.E., Garzon, M.H., var, V.H., Jakiela, M., Smith, R.E. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference GECCO 1999, Orlando, FL, vol. 1, pp. 525–532. Morgan Kaufmann Publishers, San Francisco (1999)

    Google Scholar 

  37. Pelikan, M., Sastry, K., Cant-Paz, E. (eds.): Scalable Optimization via Probabilistic Modeling. Studies in Computational Intelligence, vol. 33. Springer, Heidelberg (2006)

    MATH  Google Scholar 

  38. Sastry, K., Pelikan, M., Goldberg, D.E.: Efficiency Enhancement of Estimation of Distribution Algorithms. In: Pelikan, M., Sastry, K., Cant-Paz, E. (eds.) Scalable Optimization via Probabilistic Modeling. Studies in Computational Intelligence, vol. 33, pp. 161–186. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  39. Schäfer, J., Strimmer, K.: A Shrinkage Approach to Large-Scale Covariance Matrix Estimation and Implications for Functional Genomics. Statistical Applications in Genetics and Molecular Biology 4(1) (2005)

    Google Scholar 

  40. Shannon, C.E.: A Mathematical Theory of Communication. Bell System Technical Journal 27, 379–423 (1948)

    MATH  MathSciNet  Google Scholar 

  41. Smith, R.E., Dike, B., Stegmann, S.: Fitness inheritance in genetic algorithms. In: Proceedings of the 1995 ACM Symposium on Applied Computing, SAC 1995, pp. 345–350 (1995)

    Google Scholar 

  42. Soto, M.: Un estudio sobre los algoritmos evolutivos basados en redes bayesianas simplemente conectadas y su costo de evaluación. PhD thesis, Instituto de Cibernética, Matemática y Física, La Habana (in Spanish). Adviser: A. Ochoa (2003)

    Google Scholar 

  43. Soto, M., Ochoa, A.: A Factorized Distribution Algorithm based on polytrees. In: Congress on Evolutionary Computation, CEC 2000, California, pp. 232–237 (2000)

    Google Scholar 

  44. Soto, M., Ochoa, A., Rodríguez-Ojea, L.: An Empirical Stuty on Oversampled Selection in Estimation of Distribution Algorithms. Tech. Rep. 494, ICIMAF (2008)

    Google Scholar 

  45. Yanover, C., Weiss, Y.: Finding the M most Probable Configurations in Arbitrary Graphical Models. In: Thrun, S., Saul, L., Schollkopf, B. (eds.) Advances in Neural Information Processing Systems, vol. 16. MIT Press, Cambridge (2004)

    Google Scholar 

  46. Yunpeng, C., Xiaomin, S., Peifa, J.: Probabilistic Modeling for Continuos EDA with Boltzmann Selection and Kullback-Liebler Divergence. In: Proceedings of GECCO Conference (2006)

    Google Scholar 

  47. Zhou, Z.Z., Ong, Y.S., Nair, P.B., Keane, A.J., Lum, K.Y.: Combining global and local surrogate models to accelerate evolutionary optimization. IEEE Transactions on Systems, Man and Cybernetics-Part C 37(1), 66–76 (2007)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Mathematics and Physics, Institute of Cybernetics, Calle 15 No. 551, CP, 10400, Ciudad Habana, Cuba

    Alberto Ochoa

Authors
  1. Alberto Ochoa
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Mechanical Engineering and Science-Faculty of Engineering, Kyoto University, Yoshida-honmachi, Sakyo-Ku,, 606-8501, Kyoto, Japan

    Yoel Tenne

  2. Advanced Technology Centre, Rolls-Royce Singapore Pte Ltd, 50 Nanyang Avenue, Block N2, Level B3C,Unit 05-08, 639798, Singapore

    Chi-Keong Goh

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Ochoa, A. (2010). Opportunities for Expensive Optimization with Estimation of Distribution Algorithms. In: Tenne, Y., Goh, CK. (eds) Computational Intelligence in Expensive Optimization Problems. Adaptation Learning and Optimization, vol 2. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-10701-6_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-10701-6_8

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-10700-9

  • Online ISBN: 978-3-642-10701-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation