General multilevel adaptations for stochastic approximation algorithms of Robbins–Monro and Polyak–Ruppert type

  • Steffen Dereich
  • Thomas Müller-GronbachEmail author


In this article we present and analyse new multilevel adaptations of classical stochastic approximation algorithms for the computation of a zero of a function \(f:D \rightarrow {{\mathbb {R}}}^d\) defined on a convex domain \(D\subset {{\mathbb {R}}}^d\), which is given as a parameterised family of expectations. The analysis of the error and the computational cost of our method is based on similar assumptions as used in Giles (Oper Res 56(3):607–617, 2008) for the computation of a single expectation. Additionally, we essentially only require that f satisfies a classical contraction property from stochastic approximation theory. Under these assumptions we establish error bounds in pth mean for our multilevel Robbins–Monro and Polyak–Ruppert schemes that decay in the computational time as fast as the classical error bounds for multilevel Monte Carlo approximations of single expectations known from Giles (Oper Res 56(3):607–617, 2008). Our approach is universal in the sense that having multilevel implementations for a particular application at hand it is straightforward to implement the corresponding stochastic approximation algorithm.

Mathematics Subject Classification

Primary 62L20 Secondary 60J10 65C05 



We thank two anonymous referees for their valuable comments, which improved the presentation of the material.


  1. 1.
    Benveniste, A., Métivier, M., Priouret, P.: Adaptive Algorithms and Stochastic Approximations. Volume 22 of Applications of Mathematics (New York), vol. 22. Springer, Berlin (1990)CrossRefzbMATHGoogle Scholar
  2. 2.
    Duflo, M.: Algorithmes Stochastiques. Volume 23 of Mathématiques & Applications (Berlin) [Mathematics & Applications], vol. 23. Springer, Berlin (1996)zbMATHGoogle Scholar
  3. 3.
    Frikha, N.: Multi-level stochastic approximation algorithms. Ann. Appl. Probab. 26, 933–985 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Gaposhkin, V.F., Krasulina, T.P.: On the law of the iterated logarithm in stochastic approximation processes. Theory Probab. Appl. 19(4), 844–850 (1974)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Giles, M.B.: Multilevel Monte Carlo path simulation. Oper. Res. 56(3), 607–617 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Heinrich, S.: Multilevel Monte Carlo methods. In: Margenov, S., Waśniewski, J., Yalamov, P. (eds.) Large-Scale Scientific Computing, pp. 58–67. Springer, Berlin (2001)CrossRefGoogle Scholar
  7. 7.
    Kushner, H.J., Yang, J.: Stochastic approximation with averaging of the iterates: optimal asymptotic rate of convergence for general processes. SIAM J. Control Optim. 31(4), 1045–1062 (1993)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Kushner, H.J., Yin, G.G.: Stochastic Approximation and Recursive Algorithms and Applications, Volume 35 of Applications of Mathematics (New York). Stochastic Modelling and Applied Probability, 2nd edn. Springer, New York (2003)Google Scholar
  9. 9.
    Lai, T.L.: Stochastic approximation. Ann. Stat. 31(2), 391–406 (2003). Dedicated to the memory of Herbert E. RobbinsCrossRefzbMATHGoogle Scholar
  10. 10.
    Lai, T.L., Robbins, H.: Limit theorems for weighted sums and stochastic approximation processes. Proc. Nat. Acad. Sci. U.S.A. 75, 1068–1070 (1978)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Le Breton, A., Novikov, A.: Some results about averaging in stochastic approximation. Metrika 42(3–4):153–171 (1995). Second International Conference on Mathematical Statistics (Smolenice Castle, 1994)Google Scholar
  12. 12.
    Ljung, L., Pflug, G., Walk, H.: Stochastic Approximation and Optimization of Random Systems. Volume 17 of DMV Seminar, vol. 17. Birkhäuser Verlag, Basel (1992)CrossRefzbMATHGoogle Scholar
  13. 13.
    Nualart, D.: The Malliavin Calculus and Related Topics. Probability and Its Applications (New York), 2nd edn. Springer, Berlin (2006)zbMATHGoogle Scholar
  14. 14.
    Pelletier, M.: On the almost sure asymptotic behaviour of stochastic algorithms. Stoch. Process. Appl. 78(2), 217–244 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Pelletier, M.: Weak convergence rates for stochastic approximation with application to multiple targets and simulated annealing. Ann. Appl. Probab. 8(1), 10–44 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Polyak, B.T.: A new method of stochastic approximation type. Avtomat. i Telemekh. 51(7), 937–1008 (1998)MathSciNetzbMATHGoogle Scholar
  17. 17.
    Robbins, H., Monro, S.: A stochastic approximation method. Ann. Math. Stat. 22, 400–407 (1951)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Ruppert, D.: Almost sure approximations to the Robbins-Monro and Kiefer-Wolfowitz processes with dependent noise. Ann. Probab. 10, 178–187 (1982)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Ruppert, D.: Stochastic Approximation. In: Ghosh, B.K., Sen, P.K. (eds.) Handbook of Sequential Analysis. Volume 118 of Statist. Textbooks Monogr., pp. 503–529. Dekker, New York (1991)Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Fachbereich 10: Mathematik und Informatik, Institut für Mathematische StatistikWestfälische Wilhelms-Universität MünsterMünsterGermany
  2. 2.Fakultät für Informatik und MathematikUniversität PassauPassauGermany

Personalised recommendations