Skip to main content
SpringerLink
Log in
Book cover

Modeling Uncertainty pp 383–417Cite as

Random Search Under Additive Noise

  • Chapter

Part of the book series: International Series in Operations Research & Management Science ((ISOR,volume 46))

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Aarts, E. and J. Korst. (1989). Simulated Annealing and Boltzmann Machines, John Wiley, New York.

    MATH  Google Scholar 

  • Ackley, D.H. (1987). A Connectionist Machine for Genetic Hillclimbing, Kluwer Academic Publishers, Boston.

    Google Scholar 

  • Aluffi-Pentini, F., V. Parisi, and F. Zirilli. (1985). “Global optimization and stochastic differential equations,” Journal of Optimization Theory and Applications, vol. 47, pp. 1–16.

    MathSciNet  MATH  Google Scholar 

  • Anily, S. and A. Federgruen. (1987). “Simulated annealing methods with general acceptance probabilities,” Journal of Applied Probability, vol. 24, pp. 657–667.

    MathSciNet  MATH  Google Scholar 

  • Banzhaf, W., P. Nordin, and R. E. Keller. (1988). Genetic Programming: An Introduction: On the Automatic Evolution of Computer Programs and Its Applications, Morgan Kaufman, San Mateo, CA.

    MATH  Google Scholar 

  • Baum, L. and M. Katz. (1965). “Convergence rates in the law of large numbers,” Transactions of the American Mathematical Society, vol. 121, pp. 108–123.

    MathSciNet  MATH  Google Scholar 

  • Becker, R.W. and G. V. Lago. (1970). “A global optimization algorithm,” in: Proceedings of the 8th Annual Allerton Conference on Circuit and System Theory, pp. 3–12.

    Google Scholar 

  • Bekey, G.A. and M. T. Ung. (1974). “A comparative evaluation of two global search algorithms,” IEEE Transactions on Systems, Man and Cybernetics, vol. SMC-4, pp. 112–116.

    MATH  Google Scholar 

  • G. L. Bilbro, G.L. and W. E. Snyder. (1991). “Optimization of functions with many minima,” IEEE Transactions on Systems, Man and Cybernetics, vol. SMC-21, pp. 840–849.

    MathSciNet  Google Scholar 

  • Boender, C.G.E., A. H. G. Rinnooy Kan, L. Stougie, and G. T. Timmer. (1982). “A stochastic method for global optimization,” Mathematical Programming, vol. 22, pp. 125–140.

    MathSciNet  MATH  Google Scholar 

  • Bohachevsky, I.O. (1986). M. E. Johnson, and M. L. Stein, “Generalized simulated annealing for function optimization,” Technometrics, vol. 28, pp. 209–217.

    MATH  Google Scholar 

  • Bremermann, H.J. (1962). “Optimization through evolution and recombination,” in: Self-Organizing Systems, (edited by M. C. Yovits, G. T. Jacobi and G. D. Goldstein), pp. 93–106, Spartan Books, Washington, D.C.

    Google Scholar 

  • Bremermann, H.J. (1968). “Numerical optimization procedures derived from biological evolution processes,” in: Cybernetic Problems in Bionics, (edited by H. L. Oestreicher and D. R. Moore), pp. 597–616, Gordon and Breach Science Publishers, New York.

    Google Scholar 

  • Brooks, S.H. (1958). “A discussion of random methods for seeking maxima,” Operations Research, vol. 6, pp. 244–251.

    Google Scholar 

  • Brooks, S.H. (1959). “A comparison of maximum-seeking methods,” Operations Research, vol. 7, pp. 430–457.

    Google Scholar 

  • Bäck, T., F. Hoffmeister, and H.-P. Schwefel. (1991). “A survey of evolution strategies,” in: Proceedings of the Fourth International Conference on Ge-netic Algorithms, (edited by R. K. Belew and L. B. Booker), pp. 2–9, Morgan Kaufman Publishers, San Mateo, CA.

    Google Scholar 

  • Cerny, V. (1985). “Thermodynamical approach to the traveling salesman problem: an efficient simulation algorithm,” Journal of Optimization Theory and Applications, vol. 45, pp. 41–51.

    MathSciNet  MATH  Google Scholar 

  • Dekkers, A. and E. Aarts. (1991). “Global optimization and simulated annealing,” Mathematical Programming, vol. 50, pp. 367–393.

    MathSciNet  MATH  Google Scholar 

  • Devroye, L. (1972). “The compound random search algorithm,” in: Proceedings of the International Symposium on Systems Engineering and Analysis, Purdue University, vol. 2, pp. 195–110.

    Google Scholar 

  • Devroye, L. (1976). “On the convergence of statistical search,” IEEE Transactions on Systems, Man and Cybernetics, vol. SMC-6, pp. 46–56.

    MathSciNet  MATH  Google Scholar 

  • Devroye, L. (1976). “On random search with a learning memory,” in: Proceedings of the IEEE Conference on Cybernetics and Society, Washington, pp. 704–711.

    Google Scholar 

  • L. Devroye, L. (1977). “An expanding automaton for use in stochastic optimization,” Journal of Cybernetics and Information Science, vol. 1, pp. 82–94.

    Google Scholar 

  • Devroye, L. (1978a). “The uniform convergence of nearest neighbor regression function estimators and their application in optimization,” IEEE Transactions on Information Theory, vol. IT-24, pp. 142–151.

    MathSciNet  MATH  Google Scholar 

  • Devroye, L. (1978b). “Rank statistics in multimodal stochastic optimization,” Technical Report, School of Computer Science, McGill University.

    Google Scholar 

  • Devroye, L. (1978c). “Progressive global random search of continuous functions,” Mathematical Programming, vol. 15, pp. 330–342.

    MathSciNet  MATH  Google Scholar 

  • Devroye, L. (1979). “Global random search in stochastic optimization problems,” in: Proceedings of Optimization Days 1979, Montreal.

    Google Scholar 

  • de Biase, L. and F. Frontini. (1978). “A stochastic method for global optimization: its structure and numerical performance,” in: Towards Global Optimization 2, (edited by L. C. W. Dixon and G. P. Szegö pp. 85–102, North Holland, Amsterdam.

    MATH  Google Scholar 

  • Dvoretzky, A., J. C. Kiefer, and J. Wolfowitz. (1956). “Asymptotic minimax character of the sample distribution function and of the classical multinomial estimator,” Annals of Mathematical Statistics, vol. 27, pp. 642–669.

    MathSciNet  MATH  Google Scholar 

  • Ermakov, S.M. and A. A. Zhiglyavskii. (1983). “On random search for a global extremum,” Theory of Probability and its Applications, vol. 28, pp. 136–141.

    MathSciNet  Google Scholar 

  • Ermoliev, Yu. and R. Wets. (1988). “Stochastic programming, and introduction,” in: Numerical Techniques of Stochastic Optimization, (edited by R. J.-B. Wets and Yu. M. Ermoliev), pp. 1–32, Springer-Verlag, New York.

    MATH  Google Scholar 

  • Fisher, L. and S. J. Yakowitz. (1976). “Uniform convergence of the potential function algorithm,” SIAM Journal on Control and Optimization, vol. 14, pp. 95–103.

    MathSciNet  MATH  Google Scholar 

  • Gastwirth, J.L. (1966). “On robust procedures,” Journal of the American Statistical Association, vol. 61, pp. 929–948.

    MathSciNet  MATH  Google Scholar 

  • Gaviano, M. (1975). “Some general results on the convergence of random search algorithms in minimization problems,” in: Towards Global Optimization, (edited by L. C. W. Dixon and G. P. Szegö), pp. 149–157, North Holland, New York.

    Google Scholar 

  • Geffroy, J. (1958). “Contributions à la théorie des valeurs extrêmes,” Publications de l’Institut de Statistique des Universités de Paris, vol. 7, pp. 37–185.

    MathSciNet  Google Scholar 

  • Gelfand, S.B. and S. K. Mitter. (1991). “Weak convergence of Markov chain sampling methods and annealing algorithms to diffusions,” Journal of Optimization Theory and Applications, vol. 68, pp. 483–498.

    MathSciNet  MATH  Google Scholar 

  • Geman, S. and C.-R. Hwang. (1986). “Diffusions for global optimization,” SIAM Journal on Control and Optimization, vol. 24, pp. 1031–1043.

    MathSciNet  MATH  Google Scholar 

  • Gidas, B. (1985). “Global optimization via the Langevin equation,” in: Proceedings of the 24th IEEE Conference on Decision and Control, Fort Lauderdale, pp. 774–778.

    Google Scholar 

  • Gnedenko, A.B.V. (1943). Sur la distribution du terme maximum d’une série aléatoire, Annals of Mathematics, vol. 44, pp. 423–453.

    MathSciNet  Google Scholar 

  • Goldberg, D.E. (1989). Genetic Algorithms in Search Optimization and Machine Learning, Addison-Wesley, Reading, Mass.

    MATH  Google Scholar 

  • Gurin, L.S. (1966). “Random search in the presence of noise,” Engineering Cybernetics, vol. 4, pp. 252–260.

    MathSciNet  Google Scholar 

  • Gurin, L.S. and L. A. Rastrigin. (1965). “Convergence of the random search method in the presence of noise,” Automation and Remote Control, vol. 26, pp. 1505–1511.

    MATH  Google Scholar 

  • Haario, H. and E. Saksman. (1991). “Simulated annealing process in general state space,” Advances in Applied Probability, vol. 23, pp. 866–893.

    MathSciNet  MATH  Google Scholar 

  • Hajek, B. (1988). “Cooling schedules for optimal annealing,” Mathematics of Operations Research, vol. 13, pp. 311–329.

    MathSciNet  MATH  Google Scholar 

  • Hajek, B. and G. Sasaki. (1989). “Simulated annealing—to cool or not,” Systems and Control Letters, vol. 12, pp. 443–447.

    MathSciNet  MATH  Google Scholar 

  • Holland, J.H. (1973). “Genetic algorithms and the optimal allocation of trials,” SIAM Journal on Computing, vol. 2, pp. 88–105.

    MathSciNet  MATH  Google Scholar 

  • Holland, J.H. (1992). Adaptation in Natural and Artificial Systems: An Introductory Analysis With Applications to Biology, Control, and Artificial Intelligence, MIT Press, Cambridge, Mass.

    Google Scholar 

  • Jarvis, R.A. (1975). “Adaptive global search by the process of competitive evolution,” IEEE Transactions on Systems, Man and Cybernetics, vol. SMC-5, pp. 297–311.

    MATH  Google Scholar 

  • Johnson, D.S., C. R. Aragon, L. A. McGeogh, and C. Schevon. (1989). “Optimization by simulated annealing: an experimental evaluation; part I, graph partitioning,” Operations Research, vol. 37, pp. 865–892.

    MATH  Google Scholar 

  • Rinnooy Kan, A.H.G. and G. T. Timmer. (1984). “Stochastic methods for global optimization,” American Journal of Mathematical and Management Sciences, vol. 4, pp. 7–40.

    MathSciNet  MATH  Google Scholar 

  • Karmanov, V.G. (1974). “Convergence estimates for iterative minimization methods,” USSR Computational Mathematics and Mathematical Physics, vol. 14(1), pp. 1–13.

    MathSciNet  MATH  Google Scholar 

  • Kiefer, J. and J. Wolfowitz. (1952). “Stochastic estimation of the maximum of a regression function,” Annals of Mathematical Statistics, vol. 23, pp. 462–466.

    MathSciNet  MATH  Google Scholar 

  • Kirkpatrick, S., C. D. Gelatt, and M. P. Vecchi. (1983). “Optimization by simulated annealing,” Science, vol. 220, pp. 671–680.

    MathSciNet  MATH  Google Scholar 

  • Koronacki, J. (1976). “Convergence of random-search algorithms,” Automatic Control and Computer Sciences, vol. 10(4), pp. 39–45.

    MathSciNet  Google Scholar 

  • Kushner, H.L. (1987). “Asymptotic global behavior for stochastic approximation via diffusion with slowly decreasing noise effects: global minimization via Monte Carlo,” SIAM Journal on Applied Mathematics, vol. 47, pp. 169–185.

    MathSciNet  MATH  Google Scholar 

  • Lai, T.L. and H. Robbins. (1985) “Asymptotically efficient adaptive allocation rules,” Advances in Applied Mathematics, vol. 6, pp. 4–22.

    MathSciNet  MATH  Google Scholar 

  • Mann, H.B. and D. R. Whitney. (1947). “On a test of whether one or two random variables is stochastically larger than the other,” Annals of Mathematical Statistics, vol. 18, pp. 50–60.

    MathSciNet  MATH  Google Scholar 

  • Marti, K. (1982). “Minimizing noisy objective functions by random search methods,” Zeitschrift für Angewandte Mathematik und Mechanik, vol. 62, pp. T377–T380.

    MathSciNet  MATH  Google Scholar 

  • Marti, K. (1992). “Stochastic optimization in structural design,” Zeitschrift für Angewandte Mathematik und Mechanik, vol. 72, pp. T452–T464.

    MathSciNet  MATH  Google Scholar 

  • Massart, P. (1990). “The tight constant in the Dvoretzky-Kiefer-Wolfowitz inequality,” Annals of Probability, vol. 18, pp. 1269–1283.

    MathSciNet  MATH  Google Scholar 

  • Matyas, J. (1965). “Random optimization,” Automation and Remote Control, vol. 26, pp. 244–251.

    MathSciNet  MATH  Google Scholar 

  • Meerkov, S.M. (1972). “Deceleration in the search for the global extremum of a function,” Automation and Remote Control, vol. 33, pp. 2029–2037.

    MATH  Google Scholar 

  • Metropolis, N., A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller. (1953). “Equation of state calculation by fast computing machines,” Journal of Chemical Physics, vol. 21, pp. 1087–1092.

    Google Scholar 

  • Mockus, J.B. (1989). Bayesian Approach to Global Optimization, Kluwer Academic Publishers, Dordrecht, Netherlands.

    MATH  Google Scholar 

  • Männer, R. and H.-P. Schwefel. (1991). “Parallel Problem Solving from Nature,” vol. 496, Lecture Notes in Computer Science, Springer-Verlag, Berlin.

    MATH  Google Scholar 

  • Petrov, V.V. (1975). Sums of Independent Random Variables, Springer-Verlag, Berlin.

    MATH  Google Scholar 

  • Pinsky, M.A. (1991). Lecture Notes on Random Evolution, World Scientific Publishing Company, Singapore.

    Google Scholar 

  • Pintér, J. (1984). “Convergence properties of stochastic optimization procedures,” Mathematische Operationsforschung und Statistik, Series Optimization, vol. 15, pp. 405–427.

    MathSciNet  MATH  Google Scholar 

  • Pintér, J. (1996). Global Optimization in Action, Kluwer Academic Publishers, Dordrecht.

    MATH  Google Scholar 

  • Price, W.L. (1983). “Global optimization by controlled random search,” Journal of Optimization Theory and Applications, vol. 40, pp. 333–348.

    MathSciNet  MATH  Google Scholar 

  • Rechenberg, I. (1973). Evolutionsstrategie—Optimierung technischer Systeme nach Prinzipien der biologischen Evolution, Frommann-Holzboog, Stuttgart.

    Google Scholar 

  • Rinnooy Kan, A.H.G. and G. T. Timmer. (1987). “Stochastic global optimization methods part II: multi level methods,” Mathematical Programming, vol. 39, pp. 57–78.

    MathSciNet  MATH  Google Scholar 

  • Rinnooy Kan, A.H.G. and G. T. Timmer. (1987). “Stochastic global optimization methods part I: clustering methods,” Mathematical Programming, vol. 39, pp. 27–56.

    MathSciNet  MATH  Google Scholar 

  • Robbins, H. (1952). “Some aspects of the sequential design of experiments,” Bulletin of the American Mathematical Society, vol. 58, pp. 527–535.

    MathSciNet  MATH  Google Scholar 

  • Rubinstein, R. Y. and I. Weissman. (1979). “The Monte Carlo method for global optimization,” Cahiers du Centre d’Etude de Recherche Operationelle, vol. 21, pp. 143–149.

    MathSciNet  MATH  Google Scholar 

  • Schumer, M.A. and K. Steiglitz. (1968). “Adaptive step size random search,” IEEE Transactions on Automatic Control, vol. AC-13, pp. 270–276.

    Google Scholar 

  • Schwefel, H.-P. (1977). Modellen mittels der Evolutionsstrategie, Birkhäuser Verlag, Basel.

    MATH  Google Scholar 

  • Schwefel, H.-P. (1981). Numerical Optimization of Computer Models, John Wiley, Chichester.

    MATH  Google Scholar 

  • Schwefel, H.-P. (1995). Evolution and Optimum Seeking, Wiley, New York.

    MATH  Google Scholar 

  • Sechen, C. (1988). VLSI Placement and Global Routing using Simulated Annealing, Kluwer Academic Publishers.

    Google Scholar 

  • Shorack, G.R. and J. A. Wellner. (1986). Empirical Processes with Applications to Statistics, John Wiley, New York.

    MATH  Google Scholar 

  • Shubert, B.O. (1972). “A sequential method seeking the global maximum of a function,” SIAM Journal on Numerical Analysis, vol. 9, pp. 379–388.

    MathSciNet  MATH  Google Scholar 

  • F. J. Solis, F.J. and R. B. Wets. (1981). “Minimization by random search techniques,” Mathematics of Operations Research, vol. 1, pp. 19–30.

    MathSciNet  MATH  Google Scholar 

  • Tarasenko, G.S. (1977). “Convergence of adaptive algorithms of random search,” Cybernetics, vol. 13, pp. 725–728.

    Google Scholar 

  • Törn, A. (1974). Global Optimization as a Combination of Global and Local Search, Skriftserie Utgiven av Handelshogskolan vid Abo Akademi, Abo, Finland.

    Google Scholar 

  • Törn, A. (1976). “Probabilistic global optimization, a cluster analysis approach,” in: Proceedings of the EURO II Conference, Stockholm, Sweden, pp. 521–527, North Holland, Amsterdam.

    Google Scholar 

  • Törn, A. and A. Žilinskas. (1989). Global Optimization, Lecture Notes in Computer Science, vol. 350, Springer-Verlag, Berlin.

    MATH  Google Scholar 

  • Uosaki, K., H. Imamura, M. Tasaka, and H. Sugiyama. (1970). “A heuristic method for maxima searching in case of multimodal surfaces,” Technology Reports of Osaka University, vol. 20, pp. 337–344.

    Google Scholar 

  • Vanderbilt, D. and S. G. Louie. (1984). “A Monte Carlo simulated annealing approach to optimization over continuous variables,” Journal of Computational Physics, vol. 56, pp. 259–271.

    MathSciNet  MATH  Google Scholar 

  • Van Laarhoven, P.J.M. and E. H. L. Aarts. (1987). Simulated Annealing: Theory and Applications, D. Reidel, Dordrecht.

    MATH  Google Scholar 

  • Wasan, M.T. (1969). Stochastic Approximation, Cambridge University Press, New York.

    MATH  Google Scholar 

  • Wilcoxon, F. (1945). “Individual comparisons by ranking methods,” Biometrics Bulletin, vol. 1, pp. 80–83.

    Google Scholar 

  • Yakowitz, S. (1992). “Automatic learning: theorems for concurrent simulation and optimization,” in: 1992 Winter Simulation Conference Proceedings, (edited by J. J. Swain, D. Goldsman, R. C. Crain and J. R. Wilson), pp. 487–493, ACM, Baltimore, MD.

    Google Scholar 

  • Yakowitz, S.J. (1989). “A statistical foundation for machine learning, with application to go-moku,” Computers and Mathematics with Applications, vol. 17, pp. 1095–1102.

    MathSciNet  MATH  Google Scholar 

  • Yakowitz, S.J. (1989). “A globally-convergent stochastic approximation,” Technical Report, Systems and Industrial Engineering Department, University of Arizona, Tucson, AZ.

    MATH  Google Scholar 

  • Yakowitz, S.J. (1989). “On stochastic approximation and its generalizations,” Technical Report, Systems and Industrial Engineering Department, University of Arizona, Tucson, AZ, 1989.

    MATH  Google Scholar 

  • Yakowitz, S.J. (1992). “A decision model and methodology for the AIDS epidemic,” Applied Mathematics and Computation, vol. 55, pp. 149–172.

    MathSciNet  MATH  Google Scholar 

  • Yakowitz, S.J. (1993). “Global stochastic approximation,” SIAM Journal on Control and Optimization, vol. 31, pp. 30–40.

    MathSciNet  MATH  Google Scholar 

  • Yakowitz, S.J. and L. Fisher. (1973). “On sequential search for the maximum of an unknown function,” Journal of Mathematical Analysis and Applications, vol. 41, pp. 234–259.

    MathSciNet  MATH  Google Scholar 

  • Yakowitz, S.J., R. Hayes, and J. Gani. (1992). “Automatic learning for dynamic Markov fields, with applications to epidemiology,” Operations Research, vol. 40, pp. 867–876.

    MATH  Google Scholar 

  • Yakowitz, S.J., T. Jayawardena, and S. Li. (1992). “Theory for automatic learning under Markov-dependent noise, with applications,” IEEE Transactions on Automatic Control, vol. AC-37, pp. 1316–1324.

    MATH  Google Scholar 

  • Yakowitz, S.J. and M. Kollier. (1992). “Machine learning for blackjack counting strategies,” Journal of Forecasting and Statistical Planning, vol. 33, pp. 295–309.

    MathSciNet  MATH  Google Scholar 

  • Yakowitz, S.J. and W. Lowe. (1991). “Nonparametric bandit methods,” Annals of Operations Research, vol. 28, pp. 297–312.

    MathSciNet  MATH  Google Scholar 

  • Yakowitz, S.J. and E. Lugosi. (1990). “Random search in the presence of noise, with application to machine learning,” SIAM Journal on Scientific and Statistical Computing, vol. 11, pp. 702–712.

    MathSciNet  MATH  Google Scholar 

  • Yakowitz, S.J. and A. Vesterdahl. (1993). “Contribution to automatic learning with application to self-tuning communication channel,” Technical Report, Systems and Industrial Engineering Department, University of Arizona.

    Google Scholar 

  • Zhigljavsky, A.A. (1991). Theory of Global Random Search, Kluwer Academic Publishers, Hingham, MA.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science, McGill University, Montreal, Canada, H3A 2K6

    Luc Devroye

  2. Department of Computer Science, Concordia University, Montreal, Canada, H3G 1M8

    Adam Krzyzak

Authors
  1. Luc Devroye
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Adam Krzyzak
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. University of Arizona, USA

    Moshe Dror  & Ferenc Szidarovszky  & 

  2. Université de Montréal, France

    Pierre L’Ecuyer

Rights and permissions

Reprints and permissions

Copyright information

© 2002 Springer Science + Business Media, Inc.

About this chapter

Cite this chapter

Devroye, L., Krzyzak, A. (2002). Random Search Under Additive Noise. In: Dror, M., L’Ecuyer, P., Szidarovszky, F. (eds) Modeling Uncertainty. International Series in Operations Research & Management Science, vol 46. Springer, New York, NY. https://doi.org/10.1007/0-306-48102-2_19

Download citation

Publish with us

Policies and ethics

Search

Navigation