Advertisement

Stochastic Approximation Via Averaging: The Polyak’s Approach Revisited

Conference paper
Part of the Lecture Notes in Economics and Mathematical Systems book series (LNE, volume 374)

Abstract

Recursive stochastic optimization algorithms are considered in this work. A class of multistage procedures is developed. We analyze essentially the same kind of procedures as proposed in Polyak’s recent work. A quite different approach is taken and correlated noise processes are dealt with. In lieu of evaluating the second moments, the methods of weak convergence are employed and the asymptotic properties are obtained by examining a suitably scaled sequence. Under rather weak conditions, we show that the algorithm via averaging is an efficient approach in that it provides us with the optimal convergence speed. In addition, no prewhitening filters are needed.

Keywords

recursive estimation averaging stochastic approximation weak convergence. 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    H.J. Kushner and D.S. Clark, Stochastic Approximation for Constrained and Unconstrained Systems, Springer-Verlag, Berlin, 1978.Google Scholar
  2. [2]
    G. Ch. Pflug, Stochastic minimization with constant step-size: asymptotic laws, SIAM J. Control Optim., 24 (1986), 655–666.CrossRefGoogle Scholar
  3. [3]
    H.J. Kushner, Approximation and Weak Convergence Methods for Random Processes with Applications to Stochastic Systems Theory, MIT Press, Cambridge, MA, 1984.Google Scholar
  4. [4]
    J.N. Tsitsiklis, D.P. Bertsekas and M. Athans, Distributed asynchronous deterministic and stochastic gradient optimization algorithms, IEEE Trans. Automat. Control, AC-31 (1986), 803–812.CrossRefGoogle Scholar
  5. [5]
    D.P. Bertsekas and J.N. Tsitsiklis, Parallel and Distributed Computing, Prentice-Hall, New Jersey, 1989.Google Scholar
  6. [6]
    H.J. Kushner and G. Yin, Asymptotic properties of distributed and communicating stochastic approximation algorithms, SIAM J. Control Optim., 25 (1987), 1266–1290.CrossRefGoogle Scholar
  7. [7]
    H.J. Kushner and G. Yin, Stochastic approximation algorithms for parallel and distributed processing, Stochastics, 22 (1987), 219–250.CrossRefGoogle Scholar
  8. [8]
    G. Yin and Y.M. Zhu, On w.p.l convergence of a parallel stochastic approximation algorithm, Probab. Eng. Inform. Sci., 3 (1989), 55–75.CrossRefGoogle Scholar
  9. [9]
    G. Yin, Recent progress in parallel stochastic approximations, to appear in Statistical Theory of Identification and Adaptive Control, (P. E. Caines and L. Gerencsér Eds.), Springer-Verlag, 1991.Google Scholar
  10. [10]
    B. T. Polyak, New method of stochastic approximation type, Automat. Remote Control 51 (1990), 937–946.Google Scholar
  11. [11]
    D. Ruppert, Efficient estimations from a slowly convergent Robbins-Monro process, Technical Report, No. 781, School of Oper. Res. & Industrial Eng., Cornell Univ., 1988.Google Scholar
  12. [12]
    S.N. Ethier and T.G. Kurtz, Markov Processes, Characterization and Convergence, Wiley, New York, 1986.Google Scholar
  13. [13]
    A. Benveniste, M. Metivier and P. Priouret, Adaptive Algorithms and Stochastic Approximation, Springer-Verlag, Berlin, 1990.Google Scholar
  14. [14]
    K.L. Chung, On a stochastic approximation method, Ann. Math. Statist. 25 (1954), 463–483.CrossRefGoogle Scholar
  15. [15]
    J.H. Venter, An extension of the Robbins-Monro procedure, Ann. Math. Statist. 38 (1967), 181–190.CrossRefGoogle Scholar
  16. [16]
    T.L. Lai and H. Robbins, Consistency and asymptotic efficiency of slope estimates in stochastic approximation schemes, Z. Wahrsch. verw. Gebiete 56 (1981), 329–360.CrossRefGoogle Scholar
  17. [17]
    C.Z. Wei, Multivariate adaptive stochastic approximation, Ann. Statist. 15 (1987), 1115–1130.CrossRefGoogle Scholar
  18. [18]
    G. Yin, A stopping rule for the Robbins-Monro method, J. Optim. Theory Appl., 67 (1990), 151–173.CrossRefGoogle Scholar
  19. [19]
    H.J. Kushner, Stochastic approximation with discontinuous dynamics and state dependent noise: w.p.l and weak convergence, J. Math. Anal. Appl. 82 (1981), 527–542.CrossRefGoogle Scholar
  20. [20]
    W.F. Stout, Almost sure convergence, Academic Press, London, 1974.Google Scholar
  21. [21]
    P. Billingsley, Convergence of Probability Measure, wiley, New York, 1968.Google Scholar
  22. [22]
    H.J. Kushner and J. Yang, manuscript in preparation, 1991.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1992

Authors and Affiliations

  • G. Yin
    • 1
  1. 1.Department of MathematicsWayne State UniversityDetroitUSA

Personalised recommendations