Abstract
Real-world optimization problems often are non-convex, non-differentiable and highly multimodal, which is why stochastic, population-based metaheuristics are frequently applied. If the optimization problem is also computationally very expensive, only relatively few function evaluations can be afforded. We develop a model-assisted optimization approach as a coupling of Gaussian Process modeling, a regression technique from machine learning, with the Particle Swarm Optimization metaheuristic. It uses earlier function evaluations to predict areas of improvement and exploits the model information in the heuristic search. Under the assumption of a costly target function, it is shown that model-assistance improves the performance across a set of standard benchmark functions. In return, it is possible to reduce the number of target function evaluations to reach a certain fitness level to speed up the search.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Eiben, A., Smith, J.: Introduction to evolutionary computing. Springer, Heidelberg (2003)
Kennedy, J., Eberhart, R.: Particle swarm optimization. In: IEEE Int. Conf. on Neural Networks, Perth, Australia (1995)
Engelbrecht, A.: Computational Intelligence: An Introduction. Halsted Press, New York (2002)
Büche, D., Schraudolph, N.N., Koumoutsakos, P.: Accelerating evolutionary algorithms with gaussian process fitness function models. IEEE Transactions on Systems, Man and Cybernetics 35, 183–194 (2004)
Ulmer, H., Streichert, F., Zell, A.: Evolution strategies assisted by Gaussian processes with improved preselection criterion. In: The 2003 Congress on Evolutionary Computation, CEC 2003, vol. 1 (2003)
Li, M., Li, G., Azarm, S.: A Kriging Metamodel Assisted Multi-Objective Genetic Algorithm for Design Optimization. J. of Mechanical Design 130, 031401 (2008)
Hendtlass, T.: Fitness estimation and the particle swarm optimisation algorithm. In: IEEE Congress on Evolutionary Computation (CEC), pp. 4266–4272 (2007)
Praveen, C., Duvigneau, R.: Low cost PSO using metamodels and inexact pre-evaluation: Application to aerodynamic shape design. Computer Methods in Applied Mechanics and Engineering 198(9-12), 1087–1096 (2009)
Clerc, M., Kennedy, J.: The particle swarm - explosion, stability, and convergence in a multidimensional complex space. IEEE Transactions on Evolutionary Computation 6(1), 58–73 (2002)
Kennedy, J., Mendes, R.: Population structure and particle swarm performance. In: Proc. of the Congress on Evolutionary Computation (CEC), pp. 1671–1676 (2002)
Clerc, M.: Particle Swarm Optimization. ISTE Ltd. (2005)
Janson, S., Middendorf, M.: A hierarchical particle swarm optimizer. In: Proc. of the Congress on Evolutionary Computation (CEC), vol. 2, pp. 770–776 (2003)
Williams, C.K.I.: Gaussian processes. In: Arbib, M.A. (ed.) Handbook of Brain Theory and Neural Networks, 2nd edn., pp. 466–470. MIT Press, Cambridge (2002)
Quiñonero-Candela, J., Rasmussen, C.E.: A unifying view of sparse approximate Gaussian process regression. J. of Machine Learning Research 6, 1939–1959 (2005)
Suganthan, P., Hansen, N., Liang, J., Deb, K., Chen, Y., Auger, A., Tiwari, S.: Problem definitions and evaluation criteria for the CEC 2005 special session on real-parameter optimization. Technical report, KanGAL Report No. 2005005 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kronfeld, M., Zell, A. (2010). Gaussian Process Assisted Particle Swarm Optimization. In: Blum, C., Battiti, R. (eds) Learning and Intelligent Optimization. LION 2010. Lecture Notes in Computer Science, vol 6073. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13800-3_11
Download citation
DOI: https://doi.org/10.1007/978-3-642-13800-3_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-13799-0
Online ISBN: 978-3-642-13800-3
eBook Packages: Computer ScienceComputer Science (R0)