Skip to main content
SpringerLink
Log in

Stochastic Approximation

  • Chapter
Optimization of Stochastic Models

Part of the book series: The Kluwer International Series in Engineering and Computer Science ((SECS,volume 373))

Abstract

This chapter deals with algorithms for the optimization of simulated systems.In particular we study stochastic variants of the gradient algorithm

$$x_{n + 1} = \,x_n \, - \,a_n \nabla F(x_n )]$$

which was introduced in (1.27) to solve the optimization problem

$$[F(x) = \left\| \begin{gathered} Minimize F(x) \hfill \\ \,x\, \in \,\mathbb{R}^d \hfill \\ \end{gathered} \right.]$$

where F is bounded from below.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Bibliography

  1. Benveniste A., Metivier M., Priouret P. (1990). Adaptive Algorithms and Stochastic Approximation. Springer Verlag, Berlin.

    Google Scholar 

  2. Berger R.L., Gupta S.S. (1979). Minimax subset selection rules with applications to unequal variance (unequal sample size) problems. Scand. J. Statist 7, 21–26.

    Google Scholar 

  3. Berger E. (1986). Asymptotic behaviour of a class of stochastic approximation procedures. Probab. Th. Rei. Fields 71, 517–552.

    Article  Google Scholar 

  4. Billingsley P. (1968). Convergence of probability measures. J. Wiley and Sons, New York.

    Google Scholar 

  5. Blum J.R. (1954). Approximation methods which converge with probability one. Ann. Math. Statist. 25, 382–386.

    Article  Google Scholar 

  6. Blum J.R. (1954). Multidimensional stochastic approximation methods. Ann. Math. Statist. 25, 737–744.

    Article  Google Scholar 

  7. Chen G.C., Lai T.L., Wei C.Z. (1981). Convergence systems and strong consistency of least squares estimates in regression models. J. Multivariate Anal. 11, 319–333.

    Article  Google Scholar 

  8. Chow Y.S., Robbins H. (1965). On the asymptotic theory of fixed-width sequential confidence intervals. Ann. Math. Statist. 36, 457–462.

    Article  Google Scholar 

  9. Clark D.S. (1984). Necessary and sufficient conditions for the RobbinsMonro method. Stochastic Process. Appl. 17, 359–367.

    Article  Google Scholar 

  10. Delyon B., Juditsky A. (1992). Stochastic Optimization with averaging of trajectories. Stochastics and Stochastics Reports, 39, 107–118.

    Google Scholar 

  11. Duflo M. (1990). Methodes recursiues aleatoires. Masson, Paris. 317.

    Google Scholar 

  12. Dupac V., Herkenrath U. (1982). Stochastic Approximation on a discrete set and the multi-armed bandit problem. Commun. Statist. Sequential Analysis 1 (1), 1–25.

    Article  Google Scholar 

  13. Dupac V., Herkenrath U. (1984). On integer stochastic approximation. Applikace Matematiky 29 (5), 372–383.

    Google Scholar 

  14. Dupuis P. (1987). Large deviations analysis of reflected diffusions and constrained stochastic approcimation algorithms in convex sets. Stochastics 21, 63–96.

    Google Scholar 

  15. Dupuis P., Kushner H.J. (1985). Stochastic approximations via large deviations: asymptotic properties. SIAM J. Control Optimization 23, 675 696.

    Google Scholar 

  16. Dupuis P., Kushner H.J. (1987). Asymptotic behaviour of constrained stochastic approximations via the theory of large deviations. Probab. Th. Rel. Fields 75, 223–274.

    Article  Google Scholar 

  17. Fabian V. (1967). Stochastic approximation of minima with improved asymptotic speed. Ann. Math. Statist. 38, 191–200.

    Article  Google Scholar 

  18. Fabian V. (1968). On asymptotic normality in stochastic approximation. Ann. Math. Statist. 39, 1327–1332.

    Article  Google Scholar 

  19. Fabian V. (1973). Asymptotically efficient stochastic approximation; the RM case. Ann. Statist. 1, 486–495.

    Article  Google Scholar 

  20. Farrell, R.H. (1962). Bounded length confidence intervals for the zero of a regression function. Ann. Math. Statist. 33 (2), 237–247.

    Article  Google Scholar 

  21. Fedorov V. (1972). Theory of optimal experiment. Academic press, New York.

    Google Scholar 

  22. Freidlin M.I. (1978). The averaging principle and theorems on large deviations. Russian Math. Surveys 33, 117–176.

    Article  Google Scholar 

  23. Freidlin M.I., Wentzell A.D. (1984). Random Perturbations of Dynamical Systems. Springer Verlag.

    Google Scholar 

  24. Gaivoronski A., Messina E. (1996). Optimization of stationary behavior of general stochastic Discrete Event Systems. Preprint, Unuersita di Milano.

    Google Scholar 

  25. Gibbons J.D., Olkin I., Sobel M. (1977). Selecting and Ordering Populations. Wiley, New York.

    Google Scholar 

  26. Gittins J.C. (1979). Bandit processes and dynamic allocation indices. J. Roy. Statist. Soc. 41, 148–177.

    Google Scholar 

  27. Graham A. (1981). Kronecker products and matrix calculus. Ellis Horwood.

    Google Scholar 

  28. Gupta S.S, Huang D.Y. (1976). Subset selection procedures for the means and variances of normal populations: Unequal sample sizes case. Sankhya, A36, 112–128.

    Google Scholar 

  29. Gupta S. S., Panchapakesan, S. (1979). Multiple Decision Procedures: Theory and methodology of selecting and ranking populations. Wiley, New York.

    Google Scholar 

  30. Hale J. (1977). Theory of Functional Differential Equations. Springer Verlag.

    Google Scholar 

  31. Herkenrath U. (1983). The N-Armed Bandit with Unimodal Structure. Metrika 30, 195–210.

    Article  Google Scholar 

  32. Hiriart-Urruty J.B. (1977). Algorithms of penalization type and dual type for the solution of stochastic optimization problems with stochastic constraints. Recent Developments in Statistics (ed. J.R. Barraet al. North-Holland 183–219.

    Google Scholar 

  33. Ho, Y.C., Sreenivas R.S., Vakili P. (1992). Ordinal Optimization of DEDS. J. of Discrete Event Dynamical Systems 2 (2), 61–88.

    Article  Google Scholar 

  34. Kendall M.G., Stuart A. (1963). The Advanced Theory of Statistics, Vol. I. Griffin, London.

    Google Scholar 

  35. Kiefer J., Wolfowitz J. (1952). Stochastic estimation of the maximum of a regression function. Ann. Math. Statist. 23, 462–466.

    Article  Google Scholar 

  36. Kleijnen J. P. C. (1986). Statistical Tools for Simulation Practitioners. Marcel Dekker, New York.

    Google Scholar 

  37. Kleijnen J. P. C., van Groenendaal W. (1992). Simulation: A statistical perspective. J. Wiley and Sons, New York

    Google Scholar 

  38. Kushner H.J. (1972). Stochastic approximation algorithms for the local optimization of functions with nonunique stationary points. IEEE Trans. Automatic Control AC-17, 646–654.

    Google Scholar 

  39. Kushner H.J. (1984). Asymptotic behaviour of stochastic approximation and large deviations. IEEE Trans. Automatic Control AC-29, 984–990.

    Google Scholar 

  40. Kushner H.J., Sanvicente E. (1975). Stochastic approximation for constrained systems with observation noise on the system and constraints. Automatica 11, 375–380.

    Article  Google Scholar 

  41. Kushner H.J., Yang J. (1993). Stochastic approximation with averaging of the iterates: Optimal asymptotic rates of convergence for general processes. SIAM. of Control and Optimization 31, 1045–1062.

    Article  Google Scholar 

  42. Lai, T.L., Robbins H. (1985). Asymptotically efficient adaptive allocationrules. Adv. ’Appl. Math. 6, 4–22.

    Article  Google Scholar 

  43. L’Ecuyer P., Yin G. (1996). Budget dependent rate of stochastic approximation. To appear in SIAM J. of Control.

    Google Scholar 

  44. Ljung L. (1977). Analysis of recursive stochastic algorithms. IEEE Trans. Automatic Control AC-22, 551–575.

    Google Scholar 

  45. Ljung L. (1978). Strong convergence of a stochastic approximation algorithm. Ann. Statist. 6, 680–696.

    Article  Google Scholar 

  46. Ljung L., Pflug G., H. Walk (1992). Stochastic Approximation and Optimization of Random Systems. Birkhäuser Verlag, Basel.

    Google Scholar 

  47. Mathai A.M., Provost S.B. (1992). Quadratic forms in Random Variables. Theory and Applications. Marcel Dekker, New York.

    Google Scholar 

  48. Marti K. (1992). Semi-Stochastic Approximation by the Response Surface Methodology. Optimization 25, 209–230.

    Article  Google Scholar 

  49. J McLeish D.L. (1976). Functional and random central limit theorems for the Robbins-Monro process. J. Appl. Probab. 13, 148–154.

    Article  Google Scholar 

  50. Metivier M., Priouret P. (1984). Applications of a Kushner and Clark Lemma to general classes of stochastic algorithms. IEEE Trans. Information Theory IT-30, 140–151.

    Google Scholar 

  51. Metivier M., Priouret P. (1987). Theoremes de convergence presque sure pour une classe d’algorithmes stochastiques a pas decroissant. Probab. Th. Rei. Fields 74, 403–428.

    Article  Google Scholar 

  52. Mukerjee H.G. (1981). A stochastic approximation by observations on a discrete lattice using isotonic regression. Ann. Statist. 9, 1020–1025.

    Article  Google Scholar 

  53. Pflug G. (1981). On the convergence of a penalty-type stochastic approximation procedure. J. Information & Optimization Sciences 2, 249–258.

    Google Scholar 

  54. Pflug G. (1985). Stochastic Minimization with constant step-size-Asymptotic laws. SIAM J. of Control, 14 (4) 655–666.

    Google Scholar 

  55. Pflug G. (1985). Stepsize Rules, Stopping Times and their Implementation in Stochastic Quasigradient Algorithms. In: Numerical Techniques for Stochastic Optimization (R. Wets, Yu Ermoliev eds.), Springer Series in Computational Mathematics 10, Springer-Verlag.

    Google Scholar 

  56. Pflug G. (1990). Non-asymptotic Confidence Bounds for Stochastic Approximation Algorithms with Constant Step Size. Monatshefte für Mathematik, 110, 297–314.

    Article  Google Scholar 

  57. Polyak B. (1991). Novi metod tipa stochasticeskoi approksmacii. Automatika i Telemechanika 7, 98–107 (in Russian)

    Google Scholar 

  58. Polyak B., Juditsky A. (1992). Acceleration of stochastic approximation by averaging. SIAM J. Control Optimization 30, 838–855.

    Article  Google Scholar 

  59. Robbins H., Monro S. (1951). A stochastic approximation method. Ann Math. Statist. 22, 400–407.

    Article  Google Scholar 

  60. Robbins H., Siegmund D. (1971). A convergence theorem for nonnegative almost supermartingales and some applications. Optimizing Methods in Statistics (ed. J.S. Rustagi). Acad. Press. 233–257.

    Google Scholar 

  61. Rockafellar R.T. (1973). A dual approach to solving nonlinear programming problems by unconstained optimization. Math. Progr. 5, 354–373.

    Article  Google Scholar 

  62. Ruppert D. (1982). Almost sure approximations to the Robbins-Monro and Kiefer-Wolfowitz processes with dependent noise. Ann. Probab. 10, 178–187.

    Article  Google Scholar 

  63. Ruppert D. (1988) Efficient estimators from a slowly convergent RobbinsMonro process. Technical Report 781, School of Operations Research and Industrial Engineering, Cornell University Ithaca, New York see also: Stochastic Approximation in: Handbook of Sequential Analysis (B.K. Gosh, P.K. Sen eds.) Marcel Dekker, New York, 1991, 503–529.

    Google Scholar 

  64. Schmetterer L. (1976). Sur quelques resultats asymptotiques pour le processus de Robbins-Monro. Annales Scientifiques de l’Universite de Clermont 58, 166–176

    Google Scholar 

  65. Schwabe R. (1986). Strong representation of an adaptive stochastic approximation procedure. Stoch. Processes Appl. 23, 115–130.

    Article  Google Scholar 

  66. Schwartz A., Berman N. (1989). Abstract stochastic approximations and applications. Stoch. Processes Appl. 31, 133–149.

    Article  Google Scholar 

  67. Sielken R.L. (1973). Stopping Times for Stochastic Approximation Procedures. Z. Wahrscheinlichkeitstheorie verw. Gebiete 27, 79–86.

    Article  Google Scholar 

  68. Stroup D.F., Braun H.I. (1982). On a New Stopping Rule for Stochastic Approximation. Z. Wahrscheinlichkeitstheorie verw. Gebiete 60, 535–554.

    Article  Google Scholar 

  69. Venter J.H. (1967). An extension of the Robbins-Monro procedure. Ann. Math. Statist. 38, 181–190.

    Article  Google Scholar 

  70. Walk H. (1977). An invariance principle for the Robbins-Monro process in a Hilbert space. Z. Wahrscheinlichkeitstheorie verw. Gebiete 39, 135 150.

    Google Scholar 

  71. Walk H. (1983–84). Stochastic iteration for a constrained optimization problem. Commun. Statist-Sequential Analysis 2, 369–385.

    Google Scholar 

  72. Walk H. (1988). Limit behaviour of stochastic approximation processes. Statistics & Decisions 6, 109–128.

    Google Scholar 

  73. Wei C.Z. (1987). Multivariate adaptive stochastic approximation Ann. Statist. 15, 1115–1130.

    Article  Google Scholar 

  74. Woodroofe M. (1972). Normal approximation and large deviations for the Robbins-Monro process. Z. Wahrscheinlichkeitstheorie verw. Gebiete 21, 329–338.

    Article  Google Scholar 

Download references

Authors
  1. George C. Pflug
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 1996 Kluwer Academic Publishers

About this chapter

Cite this chapter

Pflug, G.C. (1996). Stochastic Approximation. In: Optimization of Stochastic Models. The Kluwer International Series in Engineering and Computer Science, vol 373. Springer, Boston, MA. https://doi.org/10.1007/978-1-4613-1449-3_5

Download citation

  • DOI: https://doi.org/10.1007/978-1-4613-1449-3_5

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-1-4612-8631-8

  • Online ISBN: 978-1-4613-1449-3

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation