Skip to main content
SpringerLink
Log in

Asymptotic Convergence of Scaled Genetic Algorithms to Global Optima

A gentle introduction to the theory

  • Chapter
Frontiers of Evolutionary Computation

Part of the book series: Genetic Algorithms and Evolutionary Computation ((GENA,volume 11))

Abstract

We present a self-contained theoretical framework for a scaled genetic algorithm over the alphabet {0,1} which converges asymptotically to global optima as anticipated by Davis and Principe in analogy to the simulated annealing algorithm. The algorithm employs multiple-bit mutation, single-cut-point crossover and power-law scaled proportional fitness selection based upon an arbitrary fitness function. In order to achieve asymptotic convergence to global optima, the mutation and crossover rates have to be annealed to zero in proper fashion, and power-law scaling is used with logarithmic growth in the exponent. Our analysis shows that a large population size allows for a particularly slow annealing schedule for crossover. For the foremost described setting, a detailed listing of theoretical aspects is presented including prerequisites on inhomogeneous Markov chains. In particular, we focus on: (i) The drive towards uniform populations in a genetic algorithm. (ii) Weak and strong ergodicity of the inhomogeneous Markov chain describing the probabilistic model for the scaled algorithm. (iii) Convergence to globally optimal solutions. We discuss various generalizations and extensions of the core framework presented in this exposition such as larger alphabets or other versions of the mutation-crossover operator, in particular, the Vose-Liepins version of mutation-crossover. This refers to recent work by the author in [Theoretical Computer Science 259 (2001), 1–61] and [Technical Report 2002-2-002, Aizu University] where similar types of algorithms are considered over an arbitrary-size alphabet and convergence for arbitrary fitness function under more general conditions is shown. Finally, we present an outlook on further developments of the theory.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Aarts, E. H. L. and van Laarhoven, P. J. M. (1989). Simulated Annealing: An Introduction. Statistica Neerlandica, 43:31–52.

    MathSciNet  MATH  Google Scholar 

  • Agapie, A. (2001). Theoretical analysis of mutation-adaptive evolutionary algorithms. Evolutionary Computation, 9:127–146.

    Article  Google Scholar 

  • Anily, S. and Federgruen, A. (1987). Ergodicity in Parametric Non-Stationary Markov Chains: An Application to Simulated Annealing Methods. Operations Research, 35:867–874.

    MathSciNet  MATH  Google Scholar 

  • Aytug, H. and Koehler, G. J. (1996). Stopping Criteria for Finite Length Genetic Algorithms. INFORMS Journal on Computing, 8:183–191.

    MATH  Google Scholar 

  • Azencott, R., editor (1992). Simulated Annealing — Parallelization Techniques. John Wiley & Sons Publishers.

    Google Scholar 

  • Baker, J. E. (1987). Reducing Bias and Inefficiency in the Selection Algorithm. In Grefenstette, J. J., editor, Genetic Algorithms and Their Applications: Proceedings of the Second International Conference on Genetic Algorithms. Lawrence Erlbaum Publishers.

    Google Scholar 

  • Banzhaf, W., Francone, F. D., and Nordin, P. (1996). The Effect of Extensive Use of the Mutation Operator on Generalization in Genetic Programming using Sparse Data Sets. In Ebeling, W., Rechenberg, I., Schwefel, H. P., and Voigt, H. M., editors, 4th International. Conference. on Parallel Problem Solving from Nature (PPSN96), pages 300–309. Springer Verlag.

    Google Scholar 

  • Beyer, H.-G., Schwefel, H.-P., and Wegener, I. (2002). How to analyse evolutionary algorithms. Theoretical Computer Science, 287:101–130.

    Article  MathSciNet  MATH  Google Scholar 

  • Binder, K. (1978). Monte Carlo methods in Statistical Physics. Springer Verlag.

    Google Scholar 

  • Bremermann, H. J., Rogson, J., and Salaff, S. (1966). Global Properties of Evolution Processes. In Pattee, H. H., Edelsack, E., Fein, L., and Callahan, A., editors, Natural Automata and Useful Simulations, pages 3–42. Spartan Books, Washington DC.

    Google Scholar 

  • Catoni, O. (1990). Large Deviations for Annealing. PhD thesis, Université Paris XI.

    Google Scholar 

  • Catoni, O. (1991a). Applications of Sharp Large Deviations Estimates to Optimal Cooling Schedules. Annales de ľInstitute Henri Poincaré. Probabilités ét Statistique, 27:493–518.

    Google Scholar 

  • Catoni, O. (1991b). Sharp Large Deviations Estimates for Simulated Annealing Algorithms. Annales de ľInstitute Henri Poincaré Probabilités ét Statistique, 27:291–383.

    MATH  MathSciNet  Google Scholar 

  • Catoni, O. (1992). Rough Large Deviations Estimates for Simulated Annealing — Application to Exponential Schedules. The Annals of Probability, 20:1109–1146.

    MATH  MathSciNet  Google Scholar 

  • Cerf, R. (1996). An Asymptotic Theory for Genetic Algorithms. In Alliot, J. M., Lutton, E., Ronald, E., Schoenauer, M., and Snyers, D., editors, Artificial Evolution. European Conference AE 95. Brest, France, September 4–6, 1995. Selected Papers, volume 1063 of Lecture Notes in Computer Science, pages 37–53. Springer Verlag.

    Google Scholar 

  • Cerf, R. (1998). Asymptotic Convergence of Genetic Algorithms. Advances in Applied Probability, 30:521–550.

    MATH  MathSciNet  Google Scholar 

  • Chung, K. L. (1974). A Course in Probability Theory. Academic Press Publishers. Second edition.

    Google Scholar 

  • Davis, T. (1991). Toward an Extrapolation of the Simulated Annealing Convergence Theory onto the Simple Genetic Algorithm. PhD thesis, University of Florida.

    Google Scholar 

  • Davis, T. E. and Principe, J. C. (1991). A Simulated Annealing-like Convergence Theory for the Simple Genetic Algorithm. In Belew, R. K. and Booker, L. B., editors, Proceedings of the Fourth International Conference on Genetic Algorithms’ 91, pages 174–181. Morgan Kaufmann Publishers.

    Google Scholar 

  • Davis, T. E. and Principe, J. C. (1993). A Markov Chain Framework for the Simple Genetic Algorithm. Evolutionary Computation, 1:269–288.

    Google Scholar 

  • Droste, S., Jansen, T., Tinnefeld, K., and Wegener, I. (2003). A new framework for the valuation of algorithms for black-box optimization. In De Jong, K, Poli, R., and Rowe, J., editors, Foundations of Genetic Algorithms 7, pages 197–214. Morgan Kaufmann Publishers.

    Google Scholar 

  • Droste, S., Jansen, T., and Wegener, I. (2002). On the Analysis of the (1 + 1) Evolutionary Algorithm. Theoretical Computer Science, 276:51–81.

    Article  MathSciNet  MATH  Google Scholar 

  • Feller, W. (1968). An Introduction to Probability Theory and Its Applications. Volume 1. John Wiley & Sons Publishers. (third edition).

    Google Scholar 

  • Feller, W. (1971). An Introduction to Probability Theory and Its Applications. Volume 2. John Wiley & Sons Publishers. (second edition).

    Google Scholar 

  • Fogel, D. B. (1994). Asymptotic Convergence Properties of Genetic Algorithms and Evolutionary Programming: Analysis and Experiments. Cybernetics and Systems, 25:389–407.

    MATH  Google Scholar 

  • Goldberg, D. E. Genetic Algorithms Tutorial. Genetic Programming Conference, Stanford University, (July 13, 1997).

    Google Scholar 

  • Goldberg, D. E. (1989). Genetic Algorithms, in Search, Optimization & Machine Learning. Addison-Wesley Publishers.

    Google Scholar 

  • Goldberg, D. E. (1990). A Note on Boltzmann Tournament Selection for Genetic Algorithms and Population Oriented Simulated Annealing. Complex Systems, 4:445–460.

    MATH  Google Scholar 

  • Goldberg, D. E. and Deb, K. (1991). A Comparative Analysis of Selection Schemes used in Genetic Algorithms. In Rawlins, G. J. E., editor, Foundations of Genetic Algorithms, pages 69–93. Morgan Kaufmann Publishers.

    Google Scholar 

  • Greenwood, G. W. and Zhu, Q. J. (2001). Convergence in Evolutionary Programs with Self-Adaptation. Evolutionary Computation, 9:147–157.

    Article  Google Scholar 

  • Greub, W. (1975). Linear Algebra. Springer-Verlag.

    Google Scholar 

  • Griffiths, R. C. and Taveré, S. (1997). Computational Methods for the Coalescent. In Donnelly, P. and Taveré, S., editors, Progress in Population Genetics and Human Evolution, volume 87 of IMA Volumes in Mathematics and its Applications, pages 165–182. Springer Verlag.

    Google Scholar 

  • Hajek, B. (1988). Cooling Schedules for Optimal Annealing. Mathematics of Operations Research, 13:345–351.

    Article  MathSciNet  Google Scholar 

  • He, J. and Kang, L. (1999). On the Convergence Rates of Genetic Algorithms. Theoretical Computer Science, 229:23–39.

    Article  MathSciNet  MATH  Google Scholar 

  • Holland, J. H. (1975). Adaptation in Natural and Artificial Systems. University of Michigan Press. Extended new edition. MIT Press (1992).

    Google Scholar 

  • Isaacson, D. L. and Madsen, R. W. (1961). Markov Chains: Theory and Applications. Prentice-Hall Publishers.

    Google Scholar 

  • Jansen, T. and Wegener, I. (2001). Real royal road functions — where crossover provably is essential. In Spector, L. and Goodman, E. D., editors, GECCO 2001, Proceedings of the Genetic and Evolutionary Computation Conference, pages 375–382. Morgan Kaufmann Publisher.

    Google Scholar 

  • Knuth, D. E. (1997a). The Art of Computer Programming. Volume 3: Sorting and Searching. Addison-Wesley Publishers, Second Edition edition.

    Google Scholar 

  • Knuth, D. E. (1997b). The Art of Computer Programming. Volume 1: Fundamental Algorithms. Addison-Wesley Publishers, Third Edition edition.

    Google Scholar 

  • Knuth, D. E. (1997c). The Art of Computer Programming. Volume 2: Seminumerical Algorithms. Addison-Wesley Publishers, Third Edition edition.

    Google Scholar 

  • Koehler, G. J. (1994). A Proof of the Vose-Liepins Conjecture. Annals of Mathematics and Artificial Intelligence, 10:408–422.

    Article  MathSciNet  Google Scholar 

  • Kushner, H. J. (1987). Asymptotic Global Behavior for Stochastic Approximation and Diffusions with Slowly Decreasing Noise Effects: Global Minimization via Monte Carlo. SIAM Journal of Applied Mathematics, 47:169–185.

    Article  MATH  MathSciNet  Google Scholar 

  • Landau, S. D. and Lifschitz, E. M. (1975). Lehrbuch der thoeretischen Physik V. Statistische Physik. Berlin: kademie Verlag.

    Google Scholar 

  • Lang, S. (1968). Analysis I. Addison-Wesley Publishers.

    Google Scholar 

  • Lang, S. (1970). Linear Algebra. Addison-Wesley Publishers, 2nd edition.

    Google Scholar 

  • Leung, Y., Chen, Z.-P., Xu, Z.-B., and Leung, K.-S. (1998). Convergence Rate for Non-Binary Genetic Algorithms with Different Crossover Operators. The Chinese University of Hong Kong. (Preprint).

    Google Scholar 

  • Lozano, J. A., Larrañaga, P., Graña, M., and Albizuri, F. X. (1999). Genetic Algorithms: bridging the Convergence Gap. Theoretical Computer Science, 229:23–39.

    Article  MathSciNet  Google Scholar 

  • Lux, T. and Schornstein, S. (2002). Genetic learning as an Explanation of Stylized Facts of Foreign Exchange Markets. Manuscript, Universität Kiel, London School of Economics. pdf-file, www.bwl.uni-kiel.de/vwlinstitute/gwrp/ publications/publications.htm.

    Google Scholar 

  • Mahfoud, S. W. (1993). Finite Markov Chain Models of an Alternative Selection Strategy for Genetic Algorithms. Complex Systems, 7:155–170.

    MATH  Google Scholar 

  • Mahfoud, S. W. and Goldberg, D. E. (1992). A Genetic Algorithm for Parallel Simulated Annealing. In Männer, R. and Manderick, B., editors, Parallel Problem Solving from Nature, 2, pages 301–310. Elsevier Publishers.

    Google Scholar 

  • Mahfoud, S. W. and Goldberg, D. E. (1995). Parallel Recombinative Simulated Annealing: a Genetic Algorithm. Parallel Computing, 21:1–28.

    Article  MathSciNet  MATH  Google Scholar 

  • Márkus, A., Renner, G., and Vanza, J. (1997). Spline Interpolation with Genetic Algorithms. In Kunii, T. L., Falcidieno, B., Savchenko, V. V., and Pasko, A., editors, Proceedings of the International Conference on Shape Modeling, 1997, March 3–6, The University of Aizu, Aizu-Wakamatsu, Japan, pages 47–54. IEEE Computer Society Press.

    Google Scholar 

  • Michalewicz, Z. (1994). Genetic Algorithms + Data Structures = Evolution Programs. Springer Verlag, 2nd edition.

    Google Scholar 

  • Mitchell, M. (1996). An Introduction to Genetic Algorithms. MIT Press.

    Google Scholar 

  • Mühlenbein, H. (1997). Genetic Algorithms. In Aarts, E. and Lenstra, L., editors, Local Search in Combinatorial Optimization, pages 131–171. John Wiley & Sons Publishers

    Google Scholar 

  • Nix, A. E. and Vose, M. D. (1992). Modeling Genetic Algorithms with Markov Chains. Annals of Mathematics and Artificial Intelligence, 5:79–88.

    Article  MathSciNet  MATH  Google Scholar 

  • Nomura, T. and Shimohara, K. (2001). An Analysis of Two-Parent Recombinations for Real-Valued Chromosomes in an Infinite Population. Evolutionary Computation, 9:283–308.

    Article  Google Scholar 

  • Poli, R. and Langdon, M. (1998). Schema Theory for Genetic Programming with One-Point Crossover and Point Mutation. Evolutionary Computation, 6(3):231–252.

    Google Scholar 

  • Poli, R. (2001). Exact Schema Theory for Genetic Programming and Variable-Length Genetic Algorithms with One-Point Crossover. Genetic Programming and Evolvable Machines, 2:123–163.

    Article  MATH  Google Scholar 

  • Rudin, W. (1973). Functional Analysis. McGraw-Hill Publishers.

    Google Scholar 

  • Rudolph, G. (1994). Convergence Analysis of Canonical Genetic Algorithms. IEEE Transactions on Neural Networks, 5:96–101.

    Article  Google Scholar 

  • Rudolph, G. (1997). Convergence Properties of Evolutionary Algorithms. Hamburg: Verlag Dr. Kovac.

    Google Scholar 

  • Savchenko, V. and Schmitt, L. M. (2001). Reconstructing occlusal surfaces of teeth using a genetic algorithm with simulated annealing type selection. In Anderson, D. C. and Lee, K., editors, Proceedings of the Sixth ACM Symposium on Solid Modeling and Applications, pages 39–46. ACM Press.

    Google Scholar 

  • Schaefer, H. H. (1974). Banach Lattices and Positive Operators, volume 215 of Die Grundlehren der mathematischen Wissenschaften in Einzeldarstellungen. Springer-Verlag.

    Google Scholar 

  • Schmitt, L. M. (2001). Theory of Genetic Algorithms. Theoretical Computer Science, 259:1–61.

    Article  MATH  MathSciNet  Google Scholar 

  • Schmitt, L. M. (2002). Theory of Genetic Algorithms II.–Convergence to optima for arbitrary fitness function. Technical Report 2002-2-002, The University of Aizu. (Submitted for journal publication).

    Google Scholar 

  • Schmitt, L. M. and Kondoh, T. (2000). Optimization of Mass Distribution in Articulated Figures with Genetic Algorithms. In Hamza, M. H., editor, Proceedings of the 1ASTED International Conference “Applied Simulation and Modelling”, pages 191–197. IASTED/ACTA Press.

    Google Scholar 

  • Schmitt, L. M. and Nehaniv, C. L. (1999). The Linear Geometry of Genetic Operators with Applications to the Analysis of Genetic Drift and Genetic Algorithms using Tournament Selection. In Nehaniv, C. L., editor, Mathematical and Computational Biology: Computational Morphogenesis, Hierarchical Complexity, and Digital Evolution, An International Workshop, 1997, October 21–25, The University of Aizu, Aizu-Wakamatsu, Japan, volume 26 of Lectures on Mathematics in the Life Sciences Series, pages 147–166. American Mathematical Society.

    Google Scholar 

  • Schmitt, L. M., Nehaniv, C. L., and Fujii, R. H. (1998). Linear Analysis of Genetic Algorithms. Theoretical Computer Science, 200:101–134.

    MathSciNet  MATH  Google Scholar 

  • Seneta, E. (1981). Non-negative Matrices and Markov Chains. Springer Series in Statistics. Springer Verlag.

    Google Scholar 

  • Suzuki, J. (1997). A Further Result on the Markov Chain Model of Genetic Algorithms and Its Application to a Simulated Annealing-like Strategy. In Belew, R. K. and Vose, M. D., editors, Foundations of Genetic Algorithms 4, pages 53–72. Morgan Kaufmann Publishers.

    Google Scholar 

  • Suzuki, J. (1998). A Further Result on the Markov Chain Model of Genetic Algorithms and Its Application to a Simulated Annealing-like Strategy. IEEE Trans. on Systems, Man, and Cybernetics — Part B, 28:95–102.

    Google Scholar 

  • van Nimwegen, E., Crutchfield, J. P., and Mitchell, M. (1999). Statistical Dynamics of the Royal Road Genetic Algorithm. Theoretical Computer Science, 229:41–102

    MathSciNet  MATH  Google Scholar 

  • Vose, M. D. and Wright, A. H. (1998a). The Simple Genetic Algorithm and the Walsh Transform: Part I, Theory. Evolutionary Computation, 6:253–273.

    Google Scholar 

  • Vose, M. D. and Wright, A. H. (1998b). The Simple Genetic Algorithm and the Walsh Transform: Part II, The Inverse. Evolutionary Computation, 6:275–289.

    Google Scholar 

  • Vose, M. D. (1999a). Random Heuristic Search. Theoretical Computer Science, 229:103–142.

    Article  MATH  MathSciNet  Google Scholar 

  • Vose, M. D. (1999b). The Simple Genetic Algorithm: Foundations and Theory. MIT Press.

    Google Scholar 

  • Vose, M. D. and Liepins, G. E. (1991). Punctuated Equilibria in Genetic Search. Complex Systems, 5:31–44.

    MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, The University of Aizu, Aizu-Wakamatsu City, Fukushima Prefecture, 965-8580, Japan

    Lothar M. Schmitt

Authors
  1. Lothar M. Schmitt
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Product Soft, Inc., Pittsburgh, Pennsylvania, USA

    Anil Menon

Additional information

This work is dedicated to Joanne Watari and Makoto Yoshida in Aizu-Wakamatsu.

Rights and permissions

Reprints and permissions

Copyright information

© 2004 Kluwer Academic Publishers

About this chapter

Cite this chapter

Schmitt, L.M. (2004). Asymptotic Convergence of Scaled Genetic Algorithms to Global Optima. In: Menon, A. (eds) Frontiers of Evolutionary Computation. Genetic Algorithms and Evolutionary Computation, vol 11. Springer, Boston, MA. https://doi.org/10.1007/1-4020-7782-3_8

Download citation

  • DOI: https://doi.org/10.1007/1-4020-7782-3_8

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-1-4020-7524-7

  • Online ISBN: 978-1-4020-7782-1

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation