Abstract
Genetic algorithms are optimization techniques especially useful in functions whose nonlinearity makes an analytical optimization impossible. This kind of functions appear when using least squares estimators in nonlinear regression problems. Least squares optimizers in general, and the Levenberg-Marquardt method in particular, are iterative methods especially designed to solve this kind of problems, but the results depend on both the features of the problem and the closeness to the optimum of the starting point. In this paper we study the least squares estimator and the optimization methods that are based on it. Then we analyze those features of real-coded genetic algorithms that can be useful in the context of nonlinear regression. Special attention will be devoted to the crossover operator, and a new operator based on confidence intervals will be proposed. This crossover provides an equilibrium between exploration and exploitation of the search space, which is very adequate for this kind of problems. To analyze the fitness and robustness of the proposed crossover operator, we will use three complex nonlinear regression problems with search domains of different amplitudes and compare its performance with that of other crossover operators and with the Levenberg-Marquardt method using a multi-start scheme.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
Bibliography
J. Antonisse. A new interpretation of schema notation that overturns the binary encoding constraint. In J. David Schaffer, editor, Third International Conference on Genetic Algorithms, pages 86–91, San Mateo, 1989. Morgan Kaufmann.
C. Barron and S. Gómez. The exponential tunneling method. Technical report, IIMAS-UNAM, 1991.
K. Bennett, M. C. Ferris, and Y. E. Ioannidis. A genetic algorithm for database query optimization. In Fourth International Conference on Genetic Algorithms,pages 400–407, San Mateo, CA, 1991. Morgan Kaufmann.
S. A. Billings and K. Z. Mao. Structure detection for non-linear rational models using genetic algorithms. Technical Report 634, Department of Automatic Control and Systems Engineering, University of Sheffield, U. K., 1996.
G. P. Box, W. G. Hunter, and J. S. Hunter. Statistics for Experimenters. Wiley, New York, 1978. pp. 483–487.
J. E. J. Dennis. Non-linear least squares and equations. In D. A. H. Jacobs, editor, The State of the Art in Numerical Analysis, pages 269–312, London, 1977. Academic Press.
J. E. J. Dennis and R. B. Schnabel. Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice-Hall, New York, 1983.
L. J. Eshelman and J. D. Schaffer. Real-coded genetic algorithms and interval-schemata. In L. Darrell Whitley, editor, Foundation of Genetic Algorithms 2, pages 187C3.3.7:1—C3.3.7:8.-202, San Mateo, 1993. Morgan Kaufmann.
A. R. Gallant. Nonlinear Statistical Models. Wiley, New York, 1987.
D. E. Goldberg. Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley, New York, 1989.
D. E. Goldberg. Real-coded genetic algorithms, virtual alphabets, and blocking. Complex Systems, 5: 139–167, 1991.
J. J. Grefenstette. Incorporating Problem Specific Knowledge into Genetic Algorithms. Morgan Kaufmann, San Mateo, CA, 1987.
F. Herrera, M. Lozano, and J. L. Verdegay. Tunning fuzzy logic controllers by genetic algorithms. International Journal of Approximate Reasoning, 12: 299315, 1995.
F. Herrera, M. Lozano, and J. L. Verdegay. Tackling real-coded genetic algorithms: Operators and tools for behavioural analysis. Artificial Intelligence Review,pages 265–319,1998. Kluwer Academic Publishers. Printed in Netherlands.
J. H. Holland. Adaptation in natural and artificial systems. The University of Michigan Press, Ann Arbor, MI, 1975.
T. Johnson and P. Husbands. System identification using genetic algorithms. In Parallel Problem Solving from Nature,volume 496 of Lecture Notes in Computer Science,pages 85–89, Berlin, 1990. Springer-Verlag.
H. Kargupta and R. E. Smith. System identification with evolving polynomial networks. In Fourth International Conference on Genetic Algorithms, pages 370376. Morgan Kaufmann, 1991.
S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi. Optimization by simulated annealing. Science, 220: 671–680, 1983.
S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi. Optimization by simulated annealing. In M. A. Fischler and O. Firschein, editors, Readings in Computer Vision: Issues, Problems, Principles, and Paradigms, pages 606–615. Kaufmann, Los Altos, CA., 1987.
J. R. Koza. Genetic Programming. The MIT Press, 1992.
C. Lanczos. Applied Analysis. Englewood Cliffs. Prentice Hall, Englewood Cliffs, NJ, 1956.
K. Levenberg. A method for the solution of certain problems in least squares. Quart. Appl. Math., 2: 164–168, 1944.
A. V. Levy and S. Gómez. The tunneling method applied to global optimization. In Society for Industrial and Applied Mathematics (SIAM), pages 213–244, Philadelphia, PA, 1985.
A. V. Levy and A. Montalvo. The tunneling algorithm for the global minimization of functions. SIAM Journal on Scientific and Statistical Computing, 6 (1): 15–29, 1985.
G. E. Liepins and M. D. Vose. Characterizing crossover in genetic algorithms. Annals of Mathematics and Artificial Intelligence, 5: 27–34, 1992.
E. Malinvaud. The consistency of nonlinear regression. Ann. Math. Stat., 41: 956–969, 1970.
D. W. Marquardt. An algorithm for least-squares estimation of nonlinear parameters. Journal of the American Statistical Association, 75: 87–91, 1963.
H. Mühlebein and D. Schlierkamp-Voosen. Predictive models for breeder genetic algorithm i. continuous parameter optimization. Evolutionary Computation, 1: 25–49, 1993.
Z. Michalewicz. Genetic Algorithms + Data Structures = Evolution Programs. Springer-Verlag, New York, 1992.
H. Midi. Preliminary estimators for robust non-linear regression estimation. Journal of Applied Statistics, 26 (5): 591–600, 1999.
J. J. Moré. Levenberg-marquardt algorithm: Implementation and theory. In G. A. Watson, editor, Lecture Notes in Mathematics, number 630 in Numerical Analysis, pages 105–116, Berlin, 1977. Springer-Verlag.
J. C. Nash. Minimizing a nonlinear sum of squares function on a small computer. J. Inst. Math. Appl., 19: 231–237, 1977.
A. S. Nissesen and H. Koivisto. Identification of multivariate volterra series using genetic algorithms”. In J.T. Alander, editor, Second Nordic Workshop on Genetic Algorithms and Their Applications, pages 151–161, University of Vaasa, Finland, 1996.
N. J. Radcliffe. Equivalence class analysis of genetic algorithms. Complex Systems, 2 (5): 183–205, 1991.
N. J. Radcliffe. Non-linear genetic representations. In R. Männer and B. Manderick, editors, Second International Conference on Parallel Problem Solving from Nature, pages 259–268, Amsterdam, 1992. Elsevier Science Publishers.
R. Y. Rubinstein. Simulation and the Monte Carlo Method. Wiley series in probability and mathematical statistics. John Wiley & Sons, 1981.
D. Schlierkamp-Voosen. Strategy adaptation by competition. In Second European Congress on Intelligent Techniques and Soft Computing, pages 1270–1274, 1994.
G. A. F. Seber and C. J. Wild. Non linear regression. Wiley, 1989.
A. Wright. Genetic algorithms for real parameter optimization. In G. J. E. Rawlin, editor, Foundations of Genetic Algorithms 1, pages 205–218, San Mateo, 1991. Morgan Kaufmann.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer Science+Business Media New York
About this chapter
Cite this chapter
Ortiz-Boyer, D., Harvás-Martínez, C., Muñoz-Pérez, J. (2003). Study of Genetic Algorithms with Crossover Based on Confidence Intervals as an Alternative to Classical Least Squares Estimation Methods for Nonlinear Models. In: Metaheuristics: Computer Decision-Making. Applied Optimization, vol 86. Springer, Boston, MA. https://doi.org/10.1007/978-1-4757-4137-7_6
Download citation
DOI: https://doi.org/10.1007/978-1-4757-4137-7_6
Publisher Name: Springer, Boston, MA
Print ISBN: 978-1-4419-5403-9
Online ISBN: 978-1-4757-4137-7
eBook Packages: Springer Book Archive