Abstract
Optimization problems in engineering involve very often nonlinear functions with multiple minima or discontinuities, or the simulation of a system in order to determine its parameters. Global search methods can compute a set of points and provide alternative design answers to a problem, but are computationally expensive. A solution for the CPU dependency is parallelization, which leads to the need to control the sampling of the search space. This paper presents a parallel implementation of two free-derivative optimization methods (Nelder-Mead and Powell), combined with two restart strategies to globalize the search. The first is based on a probability density function, while the second uses a fast algorithm to uniformly sample the space. The implementation is suited to a faculty network, avoiding special hardware requirements, complex installation or coding details.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
T. G. Kolda, R. M. Lewis, and V. Torczon, “Optimization by Direct Search: New Perspectives on Some Classical and Modern Methods,”SIAM Review, vol. 45, pp. 385-482, 2003.
A. Grosso, M. Locatelli, and F. Schoen, “A Population-based Approach for Hard Global Optimization Problems based on Dissimilarity Measures,”Mathematical Programming: Series A and B, vol. 110, pp. 373-404, 2003.
M. Mitchell,An Introduction to Genetic Algorithms (Complex Adaptive Systems): The MIT Press, 1998.
P. Moscato, “On Evolution, Search, Optimization, Genetic Algorithms and Martial Arts: Towards Memetic Algorithms.,” Caltech Concurrent Computation Program C3P Report 826, 1989.
M. Pogu and J. E. Souza de Cursi, “Global Optimization by Random Perturbation of the Gradient Method with a Fixed Parameter.,”Journal of Global Optimization, vol. 5, pp. 159-180, 1994.
S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi, “Optimization by Simulated Annealing,”Science, vol. 220, pp. 671-680, 1983.
L. Bottou and N. Murata, “Stochastic Approximations and Efficient Learning,” inThe Handbook of Brain Theory and Neural Networks, M. A. Arbib, Ed. Cambridge, USA.: Massachusetts Institute of Technology, 2002.
E. Cantu-Paz, “A survey of parallel genetic algorithms.,”Calculateurs Parallèles, Reseaux et Systèmes Répartis., vol. 10, pp. 141-171, 1998.
A. Bürmen, J. Puhan, T. Tuma, I. Fajfar, and A. Nussdorfer, “Parallel simplex algorithm for circuit optimisation,” inERK 2001, Portorož, Slovenija, 2001.
J. E. Dennis and V. Torczon, “Direct search methods on parallel machines,”SIAM J. Optimization, vol. 1, pp. 448-474, 1991.
A. Lewis, “Parallel Optimisation Algorithms for Continuous Non-Linear Numerical Simulations,” inFaculty of Engineering and Information Technology. vol. Doctor Brisbane, Australia: Griffith University, 2004.
R. Lewis, V. Torczon, and M. Trosset, “Direct search methods: then and now,”Journal of Computational and Applied Mathematics, vol. 124, 2000.
R. P. Brent, “Algorithms for Minimization without Derivatives,v inAlgorithms for Minimization without Derivatives Englewood Cliffs, NJ, EUA: Prentice-Hall, 1973.
W. T. Vetterling and B. P. Flannery,Numerical Recipes in C++: The Art of Scientific Computing: Cambridge University Press, 2002.
M. Matsumoto and T. Nishimura, “Mersenne Twister: a 623-dimensionally equidistributed uniform pseudorandom number generator.,vACM Trans. on Modeling and Computer Simulation, vol. 8, pp. 3-30, 1998.
M. A. Luersen and R. L. Riche, “Globalized Nelder-Mead for engineering optimization,”Computer&Structures, vol. 82, pp. 2251-2260, 2004.
E. Parzen, “On estimation of a probability density function and mode.,”Annals of Mathematical Statistics, vol. 33, pp. 1065-1076, 1962.
D. Marshall, “Nearest Neighbour Searching in High Dimensional Metric Spaces,” inDepartment of Computer Sciences. vol. Master Thesis in Information Technology: Australian National University, 2006.
S. A. Nene and S. K. Nayar, “A simple algorithm for nearest neighbor search in high dimensions,”IEEE Transactions on Pattern Analysis and Machine Intelligence,pp. 989-1003, 1997.
J. E. Souza de Cursi and A. Koscianski, “Physically Constrained Neural Network Models for Simulation,” inAdvances and Innovations in Systems, Computing, Sciences and Software Engineering, K. Elleithy, Ed. Dordrecht, The Netherlands: Springer, 2007.
M. J. D. Powell, “Direct search algorithms for optimization calculations,”Acta Numerica, vol. 7, pp. 287-336, 1998.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer Science+Business Media B.V.
About this paper
Cite this paper
Koscianski, A., Luersen, M. (2008). Globalization and Parallelization of Nelder-Mead and Powell Optimization Methods. In: Elleithy, K. (eds) Innovations and Advanced Techniques in Systems, Computing Sciences and Software Engineering. Springer, Dordrecht. https://doi.org/10.1007/978-1-4020-8735-6_18
Download citation
DOI: https://doi.org/10.1007/978-1-4020-8735-6_18
Publisher Name: Springer, Dordrecht
Print ISBN: 978-1-4020-8734-9
Online ISBN: 978-1-4020-8735-6
eBook Packages: Computer ScienceComputer Science (R0)