Summary
This chapter describes a real-coded (i.e., continuous) Estimation of Distribution Algorithm (EDA) that solves real-valued (i.e., numerical) optimization problems of bounded difficulty quickly, accurately, and reliably. This is the real-coded Bayesian Optimization Algorithm (rBOA). The objective is to bring the power of (discrete) BOA to bear upon the area of real-valued optimization. That is, the rBOA must properly decompose a problem and effectively perform Probabilistic Building-Block Crossover (PBBC) for real-valued multivariate data. In other words, a unique feature of rBOA is to learn complex dependencies of variables and make use of mixture models at the level of substructures. To begin with, a Bayesian factorization is performed. The resulting factorization that contains linkage information is then utilized for finding implicit subproblems (i.e., substructures). Mixture models are employed for independently fitting each of these substructures. Subsequently, an independent substructure-wise sampling draws the offspring. Experimental studies show that the rBOA finds, with a sub-quadratic scale-up behavior for (additively) decomposable problems, a solution that is superior in quality to that found by advanced real-coded EDAs regardless of inherent problem characteristics. Moreover, comparable or better performance is achieved for nondecomposable problems.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
C. W. Ahn, D. E. Goldberg, and R. S. Ramakrishna. Real-coded bayesian optimization algorithm: Bringing the strength of boa into the continuous world. In enetic and Evolutionary Computation Conference — GECCO 2004, pp. 840–851, 2004.
S. Baluja and S. Davies. Using optimal dependency-trees for combinatorial optimization: Learning the structure of the search space. Technical Report CMU-CS-97-107, Carnegie Mellon University, 1997.
P. A. N. Bosman. Design and application of iterated density-estimation evolutionary algorithms. Doctoral Dissertation, Utrecht University, Utrecht, The Netherlands, 2003.
P. A. N. Bosman and D. Thierens. Optimization in continuous domains by learning and simulation of Gaussian networks. In Proceedings of OBUPM workshop at the Genectic and Evolutionary Computation Conference, pp. 208–212, 2001.
M. DeGroot. Optimal Statistical Decisions. McGraw-Hill, New York, 1970.
M. A. T. Figueiredo and A. K. Jain. Unsupervised learning of finite mixture models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(3):381–385, 2002.
G. Harik. Linkage learning in via probabilistic modeling in the ECGA. Technical Report IlliGAL-99010. University of Illinois at Urbana-Champaign, Urbana, IL, 1999.
J. Hartigan. Clustering algorithms. John Wiley & Sons, Inc., New York, 1975.
D. Heckerman, D. Geiger, and D. M. Chickering. Bayesian networks: The combination of knowledge and statistical data. Technical Report MSR-TR-94-09, Microsoft Advanced Technology Division, Microsoft Corporation, Seattle, Washington, USA, 2004.
M. Henrion. Propagating uncertainty in bayesian networks by probabilistic logic sampling. In Proceedings of the Second Conference on Uncertainty in Artificial Intelligence, pp. 149–163, 1988. Amsterdam.
P. Larrañaga, R. Etxeberria, J. A. Lozano, and J. M. Peña. Combinatorial optimization by learning and simulation of Bayesian networks. In Proceedings of the Sixteenth Conference on Uncertainty in Artificial Intelligence, pp. 343–352, 2000. Stanford.
P. Larrañaga, R. Etxeberria, J. A. Lozano, and J. M. Peña. Optimization in continuous domains by learning and simulation of Gaussian networks. In A. S. Wu, editor, Proceedings of the 2000 Genetic and Evolutionary Computation Conference Workshop Program, pp. 201–204, 2000.
P. Larrañaga and J. A. Lozano. Estimation of Distribution Algorithms. A New Tool for Evolutionary Computation. Kluwer Academic Publishers, 2002.
P. Larrañaga, J. A. Lozano, and E. Bengoetxea. Estimation of distribution algorithms based on multivariate normal and gaussian networks. Technical Report KZZA-1K-1-01, Department of Computer Science and Artificial Intelligence, University of the Basque Country, Donostia, Spain, 2001.
S. L. Lauritzen. Graphical Models. Oxford University Press, 1996.
G. McLachlan and D. Peel. Finite mixture models. John Wiley & Sons, Inc, 2000.
Z. Michalewicz. Genetic Algorithms + Data Structures = Evolution Programs. Springer-Verlag, 1992.
H. Mühlenbein and T. Mahnig. FDA — a scalable evolutionary algorithm for the optimization of additively decomposed functions. Evolutionary Computation, 7(4):353–376, 1999.
J. Ocenasek. Parallel estimation of distribution algorithms. Doctoral Dissertation, Brno University of Technology, Brno, Czech, 2002.
J. Ocenasek and J. Schwarz. Estimation of distribution algorithm for mixed continuous-discrete optimization problems. In IOS Press, editor, 2nd Euro-International Symposium on Computational Intelligence, pp. 227–232, 2002.
M. Pelikan. Bayesian optimization algorithm: From single level to hierarchy. Doctoral Dissertation, University of Illinois at Urbana-Champaign, Urbana, IL, USA, 2002.
M. Pelikan, D. E. Goldberg, and E. Cantú-Paz. BOA: The Bayesian optimization algorithm. In W. Banzhaf, J. Daida, A. E. Eiben, M. H. Garzon, V. Honavar, M. Jakiela, and R. E. Smith, editors, Proceedings of the Genetic and Evolutionary Computation Conference GECCO-99, volume 1, pp. 525–532. Morgan Kaufmann Publishers, San Francisco, CA, 1999. Orlando, FL.
M. Pelikan, D.E. Goldberg, and F.G. Lobo. A survey of optimization by building and using probabilistic models. Computational Optimization and Applications, 21(1):5–20, 2002.
H. H. Rosenbrock. An automatic method for finding the greatest or least value of a function. The Computer Journal, 3(3):175–184, 1960.
R. Salomon. Evolutionary algorithms and gradient search: Simiarities and differences. IEEE Transactions on Evolutionary Computation, 2(2):45–55, 1998.
A. Törn and A. Žilinskas. Global optimization. Springer-Verlag, 1989.
S. Tsutsui, M. Pelikan, and D. E. Goldberg. Evolutionary algorithm using marginal histogram models in continuous domain. Technical Report 2001019, University of Illinois at Urbana-Champaign, Illinois Genetic Algorithms Laboratory, Urbana, IL, USA, 2001.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Ahn, C.W., Ramakrishna, R.S., Goldberg, D.E. (2006). Real-coded Bayesian Optimization Algorithm. In: Lozano, J.A., Larrañaga, P., Inza, I., Bengoetxea, E. (eds) Towards a New Evolutionary Computation. Studies in Fuzziness and Soft Computing, vol 192. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-32494-1_3
Download citation
DOI: https://doi.org/10.1007/3-540-32494-1_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29006-3
Online ISBN: 978-3-540-32494-2
eBook Packages: EngineeringEngineering (R0)