Study of Genetic Algorithms with Crossover Based on Confidence Intervals as an Alternative to Classical Least Squares Estimation Methods for Nonlinear Models

  • Domingo Ortiz-Boyer
  • César Harvás-Martínez
  • José Muñoz-Pérez
Part of the Applied Optimization book series (APOP, volume 86)


Genetic algorithms are optimization techniques especially useful in functions whose nonlinearity makes an analytical optimization impossible. This kind of functions appear when using least squares estimators in nonlinear regression problems. Least squares optimizers in general, and the Levenberg-Marquardt method in particular, are iterative methods especially designed to solve this kind of problems, but the results depend on both the features of the problem and the closeness to the optimum of the starting point. In this paper we study the least squares estimator and the optimization methods that are based on it. Then we analyze those features of real-coded genetic algorithms that can be useful in the context of nonlinear regression. Special attention will be devoted to the crossover operator, and a new operator based on confidence intervals will be proposed. This crossover provides an equilibrium between exploration and exploitation of the search space, which is very adequate for this kind of problems. To analyze the fitness and robustness of the proposed crossover operator, we will use three complex nonlinear regression problems with search domains of different amplitudes and compare its performance with that of other crossover operators and with the Levenberg-Marquardt method using a multi-start scheme.


Real coded genetic algorithms Nonlinear regression Confidence interval based crossover. 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. J. Antonisse. A new interpretation of schema notation that overturns the binary encoding constraint. In J. David Schaffer, editor, Third International Conference on Genetic Algorithms, pages 86–91, San Mateo, 1989. Morgan Kaufmann.Google Scholar
  2. C. Barron and S. Gómez. The exponential tunneling method. Technical report, IIMAS-UNAM, 1991.Google Scholar
  3. K. Bennett, M. C. Ferris, and Y. E. Ioannidis. A genetic algorithm for database query optimization. In Fourth International Conference on Genetic Algorithms,pages 400–407, San Mateo, CA, 1991. Morgan Kaufmann.Google Scholar
  4. S. A. Billings and K. Z. Mao. Structure detection for non-linear rational models using genetic algorithms. Technical Report 634, Department of Automatic Control and Systems Engineering, University of Sheffield, U. K., 1996.Google Scholar
  5. G. P. Box, W. G. Hunter, and J. S. Hunter. Statistics for Experimenters. Wiley, New York, 1978. pp. 483–487.zbMATHGoogle Scholar
  6. J. E. J. Dennis. Non-linear least squares and equations. In D. A. H. Jacobs, editor, The State of the Art in Numerical Analysis, pages 269–312, London, 1977. Academic Press.Google Scholar
  7. J. E. J. Dennis and R. B. Schnabel. Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice-Hall, New York, 1983.zbMATHGoogle Scholar
  8. L. J. Eshelman and J. D. Schaffer. Real-coded genetic algorithms and interval-schemata. In L. Darrell Whitley, editor, Foundation of Genetic Algorithms 2, pages 187C3.3.7:1—C3.3.7:8.-202, San Mateo, 1993. Morgan Kaufmann.Google Scholar
  9. A. R. Gallant. Nonlinear Statistical Models. Wiley, New York, 1987.zbMATHCrossRefGoogle Scholar
  10. D. E. Goldberg. Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley, New York, 1989.zbMATHGoogle Scholar
  11. D. E. Goldberg. Real-coded genetic algorithms, virtual alphabets, and blocking. Complex Systems, 5: 139–167, 1991.MathSciNetzbMATHGoogle Scholar
  12. J. J. Grefenstette. Incorporating Problem Specific Knowledge into Genetic Algorithms. Morgan Kaufmann, San Mateo, CA, 1987.Google Scholar
  13. F. Herrera, M. Lozano, and J. L. Verdegay. Tunning fuzzy logic controllers by genetic algorithms. International Journal of Approximate Reasoning, 12: 299315, 1995.MathSciNetzbMATHCrossRefGoogle Scholar
  14. F. Herrera, M. Lozano, and J. L. Verdegay. Tackling real-coded genetic algorithms: Operators and tools for behavioural analysis. Artificial Intelligence Review,pages 265–319,1998. Kluwer Academic Publishers. Printed in Netherlands.Google Scholar
  15. J. H. Holland. Adaptation in natural and artificial systems. The University of Michigan Press, Ann Arbor, MI, 1975.Google Scholar
  16. T. Johnson and P. Husbands. System identification using genetic algorithms. In Parallel Problem Solving from Nature,volume 496 of Lecture Notes in Computer Science,pages 85–89, Berlin, 1990. Springer-Verlag.Google Scholar
  17. H. Kargupta and R. E. Smith. System identification with evolving polynomial networks. In Fourth International Conference on Genetic Algorithms, pages 370376. Morgan Kaufmann, 1991.Google Scholar
  18. S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi. Optimization by simulated annealing. Science, 220: 671–680, 1983.MathSciNetzbMATHCrossRefGoogle Scholar
  19. S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi. Optimization by simulated annealing. In M. A. Fischler and O. Firschein, editors, Readings in Computer Vision: Issues, Problems, Principles, and Paradigms, pages 606–615. Kaufmann, Los Altos, CA., 1987.Google Scholar
  20. J. R. Koza. Genetic Programming. The MIT Press, 1992.Google Scholar
  21. C. Lanczos. Applied Analysis. Englewood Cliffs. Prentice Hall, Englewood Cliffs, NJ, 1956.Google Scholar
  22. K. Levenberg. A method for the solution of certain problems in least squares. Quart. Appl. Math., 2: 164–168, 1944.MathSciNetzbMATHGoogle Scholar
  23. A. V. Levy and S. Gómez. The tunneling method applied to global optimization. In Society for Industrial and Applied Mathematics (SIAM), pages 213–244, Philadelphia, PA, 1985.Google Scholar
  24. A. V. Levy and A. Montalvo. The tunneling algorithm for the global minimization of functions. SIAM Journal on Scientific and Statistical Computing, 6 (1): 15–29, 1985.MathSciNetzbMATHCrossRefGoogle Scholar
  25. G. E. Liepins and M. D. Vose. Characterizing crossover in genetic algorithms. Annals of Mathematics and Artificial Intelligence, 5: 27–34, 1992.MathSciNetzbMATHCrossRefGoogle Scholar
  26. E. Malinvaud. The consistency of nonlinear regression. Ann. Math. Stat., 41: 956–969, 1970.MathSciNetzbMATHCrossRefGoogle Scholar
  27. D. W. Marquardt. An algorithm for least-squares estimation of nonlinear parameters. Journal of the American Statistical Association, 75: 87–91, 1963.MathSciNetGoogle Scholar
  28. H. Mühlebein and D. Schlierkamp-Voosen. Predictive models for breeder genetic algorithm i. continuous parameter optimization. Evolutionary Computation, 1: 25–49, 1993.CrossRefGoogle Scholar
  29. Z. Michalewicz. Genetic Algorithms + Data Structures = Evolution Programs. Springer-Verlag, New York, 1992.zbMATHCrossRefGoogle Scholar
  30. H. Midi. Preliminary estimators for robust non-linear regression estimation. Journal of Applied Statistics, 26 (5): 591–600, 1999.MathSciNetzbMATHCrossRefGoogle Scholar
  31. J. J. Moré. Levenberg-marquardt algorithm: Implementation and theory. In G. A. Watson, editor, Lecture Notes in Mathematics, number 630 in Numerical Analysis, pages 105–116, Berlin, 1977. Springer-Verlag.Google Scholar
  32. J. C. Nash. Minimizing a nonlinear sum of squares function on a small computer. J. Inst. Math. Appl., 19: 231–237, 1977.MathSciNetzbMATHCrossRefGoogle Scholar
  33. A. S. Nissesen and H. Koivisto. Identification of multivariate volterra series using genetic algorithms”. In J.T. Alander, editor, Second Nordic Workshop on Genetic Algorithms and Their Applications, pages 151–161, University of Vaasa, Finland, 1996.Google Scholar
  34. N. J. Radcliffe. Equivalence class analysis of genetic algorithms. Complex Systems, 2 (5): 183–205, 1991.MathSciNetGoogle Scholar
  35. N. J. Radcliffe. Non-linear genetic representations. In R. Männer and B. Manderick, editors, Second International Conference on Parallel Problem Solving from Nature, pages 259–268, Amsterdam, 1992. Elsevier Science Publishers.Google Scholar
  36. R. Y. Rubinstein. Simulation and the Monte Carlo Method. Wiley series in probability and mathematical statistics. John Wiley & Sons, 1981.Google Scholar
  37. D. Schlierkamp-Voosen. Strategy adaptation by competition. In Second European Congress on Intelligent Techniques and Soft Computing, pages 1270–1274, 1994.Google Scholar
  38. G. A. F. Seber and C. J. Wild. Non linear regression. Wiley, 1989.Google Scholar
  39. A. Wright. Genetic algorithms for real parameter optimization. In G. J. E. Rawlin, editor, Foundations of Genetic Algorithms 1, pages 205–218, San Mateo, 1991. Morgan Kaufmann.Google Scholar

Copyright information

© Springer Science+Business Media New York 2003

Authors and Affiliations

  • Domingo Ortiz-Boyer
    • 1
  • César Harvás-Martínez
    • 1
  • José Muñoz-Pérez
    • 2
  1. 1.Department of Computer ScienceUniversity of CórdobaCórdobaSpain
  2. 2.Department of Languages and Computer ScienceUniversity of MálagaMálagaSpain

Personalised recommendations