Abstract
In this paper, we show universal lower bounds for isotropic algorithms, that hold for any algorithm such that each new point is the sum of one already visited point plus one random isotropic direction multiplied by any step size (whenever the step size is chosen by an oracle with arbitrarily high computational power). The bound is 1–O(1/d) for the constant in the linear convergence (i.e. the constant C such that the distance to the optimum after n steps is upper bounded by Cn), as already seen for some families of evolution strategies in [19,12], in contrast with 1–O(1) for the reverse case of a random step size and a direction chosen by an oracle with arbitrary high computational power. We then recall that isotropy does not uniquely determine the distribution of a sample on the sphere and show that the convergence rate in isotropic algorithms is improved by using stratified or antithetic isotropy instead of naive isotropy. We show at the end of the paper that beyond the mathematical proof, the result holds on experiments. We conclude that one should use antithetic-isotropy or stratified-isotropy, and never standard-isotropy.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Auger, A.: Convergence results for (1,λ)-SA-ES using the theory of ϕ-irreducible markov chains. Theoretical Computer Science (in press, 2005)
Auger, A., Jebalia, M., Teytaud, O.: Xse: quasi-random mutations for evolution strategies. In: Proceedings of Evolutionary Algorithms, pages 12 (2005)
Beyer, H.-G.: The Theory of Evolutions Strategies. Springer, Heidelberg (2001)
Broyden, C.G.: The convergence of a class of double-rank minimization algorithms 2, the new algorithm. j. of the inst. for math. and applications 6, 222–231 (1970)
Chazelle, B.: The discrepancy method: randomness and complexity. Cambridge University Press, New York (2000)
Conway, J.H., Sloane, N.J.: Sphere packings, lattices and groups (1998)
Fletcher, R.: A new approach to variable-metric algorithms. computer journal 13, 317–322 (1970)
Frahm, G., Junker, M.: Generalized elliptical distributions: Models and estimation. Technical Report 0037 (2003)
Goldfarb, D.: A family of variable-metric algorithms derived by variational means. mathematics of computation 24, 23–26 (1970)
Haagerup, U.: The best constants in the khintchine inequality. Studia Math. 70, 231–283 (1982)
Hansen, N., Ostermeier, A.: Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation 11(1) (2003)
Jagerskupper, J.: In between progress rate and stochastic convergence. Dagstuhl’s seminar (2006)
Literka: A remarkable monotonic property of the gamma function. Technical report (2005)
Salomon, R.: Resampling and its avoidance in genetic algorithms. In: Porto, V.W., Saravanan, N., Waagen, D., Eiben, A.E. (eds.) Evolutionary Programming VII, pp. 335–344. Springer, Berlin (1998)
Salomon, R.: The deterministic genetic algorithm: Implementation details and some results (1999)
Shanno, D.F.: Conditioning of quasi-newton methods for function minimization. mathematics of computation 24, 647–656 (1970)
Teytaud, O., Gelly, S.: General lower bounds for evolutionary algorithms, ppsn (2006)
Wang, Z., Droegemeier, K., White, L., Navon, I.M.: Application of a new adjoint newton algorithm to the 3-d arps storm scale model using simulated data. Monthly Weather Review 125(10), 2460–2478 (1997)
Witt, C., Jägersküpper, J.: Rigorous runtime analysis of a (mu+1) es for the sphere function. In: Proceedings of the 2005 Conference on Genetic and Evolutionary Computation, pp. 849–856 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Teytaud, O., Gelly, S., Mary, J. (2006). On the Ultimate Convergence Rates for Isotropic Algorithms and the Best Choices Among Various Forms of Isotropy. In: Runarsson, T.P., Beyer, HG., Burke, E., Merelo-Guervós, J.J., Whitley, L.D., Yao, X. (eds) Parallel Problem Solving from Nature - PPSN IX. PPSN 2006. Lecture Notes in Computer Science, vol 4193. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11844297_4
Download citation
DOI: https://doi.org/10.1007/11844297_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-38990-3
Online ISBN: 978-3-540-38991-0
eBook Packages: Computer ScienceComputer Science (R0)